public inbox for [email protected]
help / color / mirror / Atom feed[PATCH v17 4/7] Track collation versions for indexes.
98+ messages / 3 participants
[nested] [flat]
* [PATCH v29 4/6] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 218 +++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 264 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 208 +++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 31 +-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1655 insertions(+), 148 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index ea2be6d819..bcddca822c 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25483,7 +25483,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index b59c5697a3..527c847e27 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index 33af4ae02a..cbac0284c7 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..d6a3cae303 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1566,6 +1645,36 @@ ReleaseDeletionLock(const ObjectAddress *object)
AccessExclusiveLock);
}
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ObjectAddresses *addrs;
+ ListCell *lc;
+
+ if (list_length(collations) == 0)
+ return;
+
+ addrs = new_object_addresses();
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ add_exact_object_address(&referenced, addrs);
+ }
+
+ recordMultipleDependencies(myself, addrs->refs, addrs->numrefs,
+ DEPENDENCY_NORMAL, record_version);
+}
+
/*
* recordDependencyOnExpr - find expression dependencies
*
@@ -1588,6 +1697,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1715,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1743,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1810,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1832,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1855,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1894,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1962,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2057,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2178,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2192,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2206,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2296,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2442,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2466,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2864,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 67144aa3c9..fc5140daa6 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2336,7 +2336,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2346,7 +2346,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3665,7 +3665,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 117e3fdef7..a30e83ca39 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/rel.h"
@@ -1020,6 +1022,10 @@ index_create(Relation heapRelation,
ObjectAddress myself,
referenced;
ObjectAddresses *addrs;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1106,20 +1112,80 @@ index_create(Relation heapRelation,
/* placeholder for normal dependencies */
addrs = new_object_addresses();
- /* Store dependency on collations */
-
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- add_exact_object_address(&referenced, addrs);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic collation, we always track the dependency on the
+ * collation but only track the version if the AM relies on a stable
+ * ordering.
+ */
+ if (determ_colls)
+ {
+ IndexAmRoutine *routine;
+ bool track_version;
+
+ routine = GetIndexAmRoutineByAmId(accessMethodObjectId, false);
+ track_version = !routine->amnostablecollorder;
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1133,21 +1199,32 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /*
+ * recordDependencyOnSingleRelExpr gets rid of duplicated entries
+ */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1226,6 +1303,108 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+/* index_check_collation_version
+ * Raise a warning if the recorded and current collation version don't
+ * match.
+*/
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+/* index_check_collation_versions
+ * Check the collation version for all dependencies on the given object.
+ */
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+/* index_update_collation_version
+ * Return the current collation version for the given object.
+ */
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+/* index_update_collation_versions
+ * Record the current collation versions of all dependencies on the given
+ * object.
+ */
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -3000,6 +3179,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3646,6 +3887,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 0d70cb0c3c..93774c9d21 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -362,7 +362,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 09c30b13e8..2bd9401e6a 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,14 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +51,24 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -66,6 +77,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
max_slots,
slot_init_count,
slot_stored_count;
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -96,12 +108,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
slot_init_count = 0;
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant dependencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (isObjectPinned(referenced, dependDesc))
+ if (!ignore_systempin && isObjectPinned(referenced, dependDesc))
continue;
if (slot_init_count < max_slots)
@@ -493,7 +542,8 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
@@ -501,8 +551,7 @@ changeDependencyFor(Oid classId, Oid objectId,
* Returns the number of records updated.
*/
long
-changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+changeDependenciesOf(Oid classId, Oid oldObjectId, Oid newObjectId)
{
long count = 0;
Relation depRel;
@@ -526,14 +575,47 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
- Form_pg_depend depform;
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
+
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
- depform->objid = newObjectId;
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -636,6 +718,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 0b04dff773..f2af6212e5 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index ddeec870d8..e1016609dc 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index f9d0d67aa7..3a20dcdee5 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 96ecad02dd..96f8c67831 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 77139355e0..90535936ae 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -286,6 +287,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -387,6 +390,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -714,6 +718,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7034,7 +7042,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7070,7 +7080,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7095,7 +7160,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7134,7 +7201,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7169,7 +7238,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7200,7 +7271,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7234,7 +7307,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7274,6 +7349,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7299,6 +7376,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16360,7 +16439,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16427,6 +16507,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16455,6 +16539,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18441,6 +18540,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index e0b42e8391..623814d1c5 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if a partition, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..2966339498 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,10 +229,9 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
+long changeDependenciesOf(Oid classId, Oid oldObjectId,
Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 687509ba92..84466ab577 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10367,6 +10367,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1488bffa2b..8c54e12268 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 64c0c66859..d883ee58d1 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,14 +2065,16 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(8 rows)
+(10 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2092,14 +2094,16 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(8 rows)
+(10 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--9jxsPFA5p3P2qPhR
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v29-0005-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v28 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 218 +++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 257 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 206 +++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 31 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1648 insertions(+), 146 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..d6a3cae303 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1566,6 +1645,36 @@ ReleaseDeletionLock(const ObjectAddress *object)
AccessExclusiveLock);
}
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ObjectAddresses *addrs;
+ ListCell *lc;
+
+ if (list_length(collations) == 0)
+ return;
+
+ addrs = new_object_addresses();
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ add_exact_object_address(&referenced, addrs);
+ }
+
+ recordMultipleDependencies(myself, addrs->refs, addrs->numrefs,
+ DEPENDENCY_NORMAL, record_version);
+}
+
/*
* recordDependencyOnExpr - find expression dependencies
*
@@ -1588,6 +1697,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1715,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1743,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1810,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1832,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1855,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1894,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1962,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2057,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2178,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2192,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2206,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2296,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2442,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2466,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2864,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..e5155785b6 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,32 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /*
+ * recordDependencyOnSingleRelExpr gets rid of duplicated entries
+ */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1284,108 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+/* index_check_collation_version
+ * Raise a warning if the recorded and current collation version don't
+ * match.
+*/
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+/* index_check_collation_versions
+ * Check the collation version for all dependencies on the given object.
+ */
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+/* index_update_collation_version
+ * Return the current collation version for the given object.
+ */
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+/* index_update_collation_versions
+ * Record the current collation versions of all dependencies on the given
+ * object.
+ */
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2977,6 +3151,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3841,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..8137e50036 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,14 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +51,24 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +76,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +95,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant dependencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,7 +500,8 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
@@ -459,8 +509,7 @@ changeDependencyFor(Oid classId, Oid objectId,
* Returns the number of records updated.
*/
long
-changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+changeDependenciesOf(Oid classId, Oid oldObjectId, Oid newObjectId)
{
long count = 0;
Relation depRel;
@@ -485,13 +534,46 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
- depform->objid = newObjectId;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -594,6 +676,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..2966339498 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,10 +229,9 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
+long changeDependenciesOf(Oid classId, Oid oldObjectId,
Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--gclmslubawp3yaq7
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v28-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v27 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 250 +++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 +++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1650 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..5c3a84629f 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1588,6 +1667,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1685,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1713,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1780,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1802,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1825,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1864,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1932,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2027,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2116,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2148,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2162,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2176,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2266,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2412,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2436,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2834,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..5cd3af3053 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1282,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1663,14 +1821,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2977,6 +3130,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3820,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..c1e37f387c 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--o56keubw4jpehgac
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v27-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v25 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1665 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index d279842d3c..93a7584a0c 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2295,7 +2295,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2305,7 +2305,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3629,7 +3629,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index fc088d3f52..e85aeb5502 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1107,19 +1114,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1130,21 +1195,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1223,6 +1297,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1674,14 +1836,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2635,6 +2792,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -2988,6 +3156,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3616,6 +3846,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index fdc63e7dea..5b3c4ab637 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..1b6c78791e 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--lrZ03NoBR/3+SXJZ
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v25-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v27 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 250 +++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 +++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1650 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..5c3a84629f 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1588,6 +1667,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1685,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1713,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1780,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1802,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1825,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1864,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1932,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2027,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2116,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2148,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2162,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2176,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2266,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2412,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2436,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2834,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..5cd3af3053 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1282,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1663,14 +1821,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2977,6 +3130,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3820,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..c1e37f387c 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--o56keubw4jpehgac
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v27-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v27 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 250 +++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 +++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1650 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..5c3a84629f 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1588,6 +1667,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1685,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1713,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1780,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1802,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1825,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1864,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1932,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2027,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2116,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2148,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2162,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2176,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2266,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2412,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2436,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2834,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..5cd3af3053 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1282,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1663,14 +1821,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2977,6 +3130,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3820,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..c1e37f387c 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--o56keubw4jpehgac
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v27-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v16 4/7] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 2 +-
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 189 +++++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 191 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 130 +++++++++++-
src/backend/catalog/pg_type.c | 69 +++++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 31 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 ++++-
src/backend/utils/cache/relcache.c | 2 +
src/include/catalog/dependency.h | 19 +-
src/include/catalog/index.h | 2 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++++
.../regress/expected/collate.icu.utf8.out | 130 ++++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 83 ++++++++
25 files changed, 979 insertions(+), 69 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index fd1bba9959..29e4a9b73e 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21125,7 +21125,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system.
+ operating system. An empty string is returned if the version is unknown.
</para>
<para>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index c54a7c420d..7209a606fa 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 660033b9c1..c0e36ef4cd 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -78,6 +78,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -138,6 +139,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -438,6 +442,80 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version, *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1591,6 +1669,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1603,9 +1685,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1632,12 +1715,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,9 +1780,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, NULL,
+ self_addrs->refs,
self_addrs->numrefs,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1712,9 +1802,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1827,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1866,46 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /* Record collations from the type itself, or underlying in case of
+ * complex type. Note that if the direct parent is a CollateExpr
+ * node, there's no need to record the type underlying collation if
+ * any. A dependency already exists for the owning relation, and a
+ * change in the collation sort order wouldn't cause any harm as
+ * the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1794,11 +1930,13 @@ find_expr_references_walker(Node *node,
/*
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
- * a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * a simple constant. However we can save work in the most common case
+ * where the collation is "default", since we know that's pinned, if
+ * the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2026,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2115,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2147,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2161,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2175,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2265,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2411,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2435,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2833,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, NULL, referenced->numrefs,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9d9e915979..4546789446 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2304,7 +2304,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2314,7 +2314,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3638,7 +3638,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */,
+ false /* don't track versions */);
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 76fd938ce3..369b5d43f1 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -52,6 +52,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -74,6 +75,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -117,6 +119,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1025,6 +1028,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1115,21 +1122,75 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or not,
+ * removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1143,21 +1204,29 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1229,6 +1298,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+static void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2635,6 +2792,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3623,6 +3791,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 312d4d3d90..3f113499b2 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,12 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +49,47 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const char *version,
int nreferenced,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +97,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +116,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries and
+ * calling CommandCounterIncrement() if the dependencies are
+ * registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -545,6 +615,54 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenceds addresses.
+ */
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index cd56714968..fe14b6bd38 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d625d17bf4..2b7de111cd 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -634,6 +636,35 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index d82fc5ab8b..f2fc427fc8 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 60dab33fcb..18a2d72f91 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1473,7 +1478,7 @@ pg_newlocale_from_collation(Oid collid)
* NULL (if it doesn't support versions). It must not return NULL for some
* collcollate and not NULL for others.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1511,6 +1516,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index ff70326474..bdf50ffe89 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/pg_am.h"
@@ -5623,6 +5624,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 473342dad2..8858ceb3df 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -156,7 +156,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -176,17 +177,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const char *version,
int nreferenced,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index a2890c1314..c619d02465 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 97890946c5..3eb5ec4fcd 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -350,6 +350,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 44ed04dd3f..3656ea94e8 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..09512c0f66 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,136 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index ae95bb38a6..94b4daf4d6 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..c8f1a620d2 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,89 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--fUYQa+Pmc3FrFX/N
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0005-Preserve-index-dependencies-on-collation-during-.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v24 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 256 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1666 insertions(+), 157 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 3c83fe6bab..d28512a638 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2301,7 +2301,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2311,7 +2311,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3635,7 +3635,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index cdc01c49c9..210b25d147 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1119,21 +1126,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1147,21 +1210,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1240,6 +1312,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1691,14 +1851,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2652,6 +2807,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3005,6 +3171,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3633,6 +3861,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..b10fc9a241 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,48 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +104,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +123,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +528,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +579,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +644,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +724,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--wac7ysb48OaltWcw
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v24-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v25 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1665 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index d279842d3c..93a7584a0c 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2295,7 +2295,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2305,7 +2305,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3629,7 +3629,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index fc088d3f52..e85aeb5502 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1107,19 +1114,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1130,21 +1195,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1223,6 +1297,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1674,14 +1836,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2635,6 +2792,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -2988,6 +3156,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3616,6 +3846,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index fdc63e7dea..5b3c4ab637 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..1b6c78791e 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--lrZ03NoBR/3+SXJZ
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v25-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v24 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 256 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1666 insertions(+), 157 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 3c83fe6bab..d28512a638 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2301,7 +2301,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2311,7 +2311,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3635,7 +3635,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index cdc01c49c9..210b25d147 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1119,21 +1126,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1147,21 +1210,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1240,6 +1312,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1691,14 +1851,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2652,6 +2807,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3005,6 +3171,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3633,6 +3861,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..b10fc9a241 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,48 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +104,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +123,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +528,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +579,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +644,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +724,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--wac7ysb48OaltWcw
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v24-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v27 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 250 +++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 +++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1650 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..5c3a84629f 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1588,6 +1667,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1685,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1713,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1780,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1802,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1825,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1864,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1932,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2027,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2116,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2148,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2162,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2176,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2266,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2412,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2436,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2834,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..5cd3af3053 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1282,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1663,14 +1821,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2977,6 +3130,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3820,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..c1e37f387c 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--o56keubw4jpehgac
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v27-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v28 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 218 +++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 257 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 206 +++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 31 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1648 insertions(+), 146 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..d6a3cae303 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1566,6 +1645,36 @@ ReleaseDeletionLock(const ObjectAddress *object)
AccessExclusiveLock);
}
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ObjectAddresses *addrs;
+ ListCell *lc;
+
+ if (list_length(collations) == 0)
+ return;
+
+ addrs = new_object_addresses();
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ add_exact_object_address(&referenced, addrs);
+ }
+
+ recordMultipleDependencies(myself, addrs->refs, addrs->numrefs,
+ DEPENDENCY_NORMAL, record_version);
+}
+
/*
* recordDependencyOnExpr - find expression dependencies
*
@@ -1588,6 +1697,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1715,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1743,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1810,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1832,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1855,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1894,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1962,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2057,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2178,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2192,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2206,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2296,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2442,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2466,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2864,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..e5155785b6 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,32 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /*
+ * recordDependencyOnSingleRelExpr gets rid of duplicated entries
+ */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1284,108 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+/* index_check_collation_version
+ * Raise a warning if the recorded and current collation version don't
+ * match.
+*/
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+/* index_check_collation_versions
+ * Check the collation version for all dependencies on the given object.
+ */
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+/* index_update_collation_version
+ * Return the current collation version for the given object.
+ */
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+/* index_update_collation_versions
+ * Record the current collation versions of all dependencies on the given
+ * object.
+ */
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2977,6 +3151,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3841,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..8137e50036 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,14 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +51,24 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +76,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +95,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant dependencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,7 +500,8 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
@@ -459,8 +509,7 @@ changeDependencyFor(Oid classId, Oid objectId,
* Returns the number of records updated.
*/
long
-changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+changeDependenciesOf(Oid classId, Oid oldObjectId, Oid newObjectId)
{
long count = 0;
Relation depRel;
@@ -485,13 +534,46 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
- depform->objid = newObjectId;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -594,6 +676,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..2966339498 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,10 +229,9 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
+long changeDependenciesOf(Oid classId, Oid oldObjectId,
Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--gclmslubawp3yaq7
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v28-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v25 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1665 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index d279842d3c..93a7584a0c 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2295,7 +2295,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2305,7 +2305,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3629,7 +3629,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index fc088d3f52..e85aeb5502 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1107,19 +1114,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1130,21 +1195,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1223,6 +1297,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1674,14 +1836,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2635,6 +2792,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -2988,6 +3156,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3616,6 +3846,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index fdc63e7dea..5b3c4ab637 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..1b6c78791e 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--lrZ03NoBR/3+SXJZ
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v25-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v27 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 250 +++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 +++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1650 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..5c3a84629f 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1588,6 +1667,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1685,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1713,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1780,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1802,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1825,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1864,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1932,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2027,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2116,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2148,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2162,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2176,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2266,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2412,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2436,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2834,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..5cd3af3053 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1282,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1663,14 +1821,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2977,6 +3130,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3820,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..c1e37f387c 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--o56keubw4jpehgac
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v27-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v25 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1665 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index d279842d3c..93a7584a0c 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2295,7 +2295,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2305,7 +2305,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3629,7 +3629,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index fc088d3f52..e85aeb5502 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1107,19 +1114,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1130,21 +1195,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1223,6 +1297,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1674,14 +1836,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2635,6 +2792,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -2988,6 +3156,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3616,6 +3846,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index fdc63e7dea..5b3c4ab637 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..1b6c78791e 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--lrZ03NoBR/3+SXJZ
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v25-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v27 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 250 +++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 +++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1650 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..5c3a84629f 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1588,6 +1667,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1685,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1713,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1780,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1802,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1825,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1864,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1932,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2027,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2116,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2148,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2162,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2176,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2266,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2412,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2436,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2834,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..5cd3af3053 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1282,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1663,14 +1821,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2977,6 +3130,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3820,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..c1e37f387c 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--o56keubw4jpehgac
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v27-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v27 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 250 +++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 +++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1650 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..5c3a84629f 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1588,6 +1667,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1685,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1713,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1780,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1802,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1825,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1864,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1932,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2027,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2116,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2148,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2162,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2176,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2266,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2412,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2436,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2834,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..5cd3af3053 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1282,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1663,14 +1821,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2977,6 +3130,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3820,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..c1e37f387c 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--o56keubw4jpehgac
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v27-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v25 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1665 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index d279842d3c..93a7584a0c 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2295,7 +2295,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2305,7 +2305,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3629,7 +3629,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index fc088d3f52..e85aeb5502 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1107,19 +1114,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1130,21 +1195,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1223,6 +1297,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1674,14 +1836,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2635,6 +2792,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -2988,6 +3156,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3616,6 +3846,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index fdc63e7dea..5b3c4ab637 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..1b6c78791e 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--lrZ03NoBR/3+SXJZ
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v25-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v28 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 218 +++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 257 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 206 +++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 31 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1648 insertions(+), 146 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..d6a3cae303 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1566,6 +1645,36 @@ ReleaseDeletionLock(const ObjectAddress *object)
AccessExclusiveLock);
}
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ObjectAddresses *addrs;
+ ListCell *lc;
+
+ if (list_length(collations) == 0)
+ return;
+
+ addrs = new_object_addresses();
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ add_exact_object_address(&referenced, addrs);
+ }
+
+ recordMultipleDependencies(myself, addrs->refs, addrs->numrefs,
+ DEPENDENCY_NORMAL, record_version);
+}
+
/*
* recordDependencyOnExpr - find expression dependencies
*
@@ -1588,6 +1697,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1715,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1743,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1810,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1832,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1855,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1894,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1962,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2057,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2178,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2192,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2206,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2296,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2442,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2466,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2864,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..e5155785b6 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,32 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /*
+ * recordDependencyOnSingleRelExpr gets rid of duplicated entries
+ */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1284,108 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+/* index_check_collation_version
+ * Raise a warning if the recorded and current collation version don't
+ * match.
+*/
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+/* index_check_collation_versions
+ * Check the collation version for all dependencies on the given object.
+ */
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+/* index_update_collation_version
+ * Return the current collation version for the given object.
+ */
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+/* index_update_collation_versions
+ * Record the current collation versions of all dependencies on the given
+ * object.
+ */
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2977,6 +3151,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3841,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..8137e50036 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,14 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +51,24 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +76,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +95,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant dependencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,7 +500,8 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
@@ -459,8 +509,7 @@ changeDependencyFor(Oid classId, Oid objectId,
* Returns the number of records updated.
*/
long
-changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+changeDependenciesOf(Oid classId, Oid oldObjectId, Oid newObjectId)
{
long count = 0;
Relation depRel;
@@ -485,13 +534,46 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
- depform->objid = newObjectId;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -594,6 +676,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..2966339498 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,10 +229,9 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
+long changeDependenciesOf(Oid classId, Oid oldObjectId,
Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--gclmslubawp3yaq7
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v28-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v28 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 218 +++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 257 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 206 +++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 31 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1648 insertions(+), 146 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..d6a3cae303 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1566,6 +1645,36 @@ ReleaseDeletionLock(const ObjectAddress *object)
AccessExclusiveLock);
}
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ObjectAddresses *addrs;
+ ListCell *lc;
+
+ if (list_length(collations) == 0)
+ return;
+
+ addrs = new_object_addresses();
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ add_exact_object_address(&referenced, addrs);
+ }
+
+ recordMultipleDependencies(myself, addrs->refs, addrs->numrefs,
+ DEPENDENCY_NORMAL, record_version);
+}
+
/*
* recordDependencyOnExpr - find expression dependencies
*
@@ -1588,6 +1697,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1715,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1743,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1810,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1832,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1855,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1894,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1962,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2057,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2178,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2192,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2206,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2296,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2442,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2466,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2864,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..e5155785b6 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,32 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /*
+ * recordDependencyOnSingleRelExpr gets rid of duplicated entries
+ */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1284,108 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+/* index_check_collation_version
+ * Raise a warning if the recorded and current collation version don't
+ * match.
+*/
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+/* index_check_collation_versions
+ * Check the collation version for all dependencies on the given object.
+ */
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+/* index_update_collation_version
+ * Return the current collation version for the given object.
+ */
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+/* index_update_collation_versions
+ * Record the current collation versions of all dependencies on the given
+ * object.
+ */
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2977,6 +3151,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3841,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..8137e50036 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,14 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +51,24 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +76,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +95,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant dependencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,7 +500,8 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
@@ -459,8 +509,7 @@ changeDependencyFor(Oid classId, Oid objectId,
* Returns the number of records updated.
*/
long
-changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+changeDependenciesOf(Oid classId, Oid oldObjectId, Oid newObjectId)
{
long count = 0;
Relation depRel;
@@ -485,13 +534,46 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
- depform->objid = newObjectId;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -594,6 +676,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..2966339498 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,10 +229,9 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
+long changeDependenciesOf(Oid classId, Oid oldObjectId,
Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--gclmslubawp3yaq7
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v28-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v25 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1665 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index d279842d3c..93a7584a0c 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2295,7 +2295,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2305,7 +2305,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3629,7 +3629,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index fc088d3f52..e85aeb5502 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1107,19 +1114,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1130,21 +1195,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1223,6 +1297,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1674,14 +1836,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2635,6 +2792,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -2988,6 +3156,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3616,6 +3846,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index fdc63e7dea..5b3c4ab637 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..1b6c78791e 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--lrZ03NoBR/3+SXJZ
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v25-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v24 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 256 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1666 insertions(+), 157 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 3c83fe6bab..d28512a638 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2301,7 +2301,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2311,7 +2311,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3635,7 +3635,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index cdc01c49c9..210b25d147 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1119,21 +1126,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1147,21 +1210,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1240,6 +1312,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1691,14 +1851,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2652,6 +2807,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3005,6 +3171,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3633,6 +3861,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..b10fc9a241 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,48 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +104,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +123,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +528,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +579,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +644,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +724,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--wac7ysb48OaltWcw
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v24-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v25 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1665 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index d279842d3c..93a7584a0c 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2295,7 +2295,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2305,7 +2305,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3629,7 +3629,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index fc088d3f52..e85aeb5502 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1107,19 +1114,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1130,21 +1195,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1223,6 +1297,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1674,14 +1836,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2635,6 +2792,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -2988,6 +3156,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3616,6 +3846,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index fdc63e7dea..5b3c4ab637 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..1b6c78791e 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--lrZ03NoBR/3+SXJZ
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v25-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v25 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1665 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index d279842d3c..93a7584a0c 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2295,7 +2295,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2305,7 +2305,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3629,7 +3629,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index fc088d3f52..e85aeb5502 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1107,19 +1114,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1130,21 +1195,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1223,6 +1297,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1674,14 +1836,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2635,6 +2792,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -2988,6 +3156,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3616,6 +3846,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index fdc63e7dea..5b3c4ab637 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..1b6c78791e 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--lrZ03NoBR/3+SXJZ
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v25-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v25 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1665 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index d279842d3c..93a7584a0c 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2295,7 +2295,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2305,7 +2305,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3629,7 +3629,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index fc088d3f52..e85aeb5502 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1107,19 +1114,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1130,21 +1195,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1223,6 +1297,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1674,14 +1836,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2635,6 +2792,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -2988,6 +3156,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3616,6 +3846,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index fdc63e7dea..5b3c4ab637 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..1b6c78791e 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--lrZ03NoBR/3+SXJZ
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v25-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v27 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 250 +++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 +++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1650 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..5c3a84629f 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1588,6 +1667,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1685,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1713,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1780,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1802,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1825,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1864,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1932,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2027,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2116,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2148,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2162,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2176,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2266,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2412,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2436,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2834,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..5cd3af3053 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1282,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1663,14 +1821,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2977,6 +3130,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3820,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..c1e37f387c 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--o56keubw4jpehgac
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v27-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v25 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1665 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index d279842d3c..93a7584a0c 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2295,7 +2295,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2305,7 +2305,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3629,7 +3629,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index fc088d3f52..e85aeb5502 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1107,19 +1114,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1130,21 +1195,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1223,6 +1297,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1674,14 +1836,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2635,6 +2792,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -2988,6 +3156,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3616,6 +3846,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index fdc63e7dea..5b3c4ab637 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..1b6c78791e 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--lrZ03NoBR/3+SXJZ
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v25-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v24 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 256 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1666 insertions(+), 157 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 3c83fe6bab..d28512a638 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2301,7 +2301,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2311,7 +2311,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3635,7 +3635,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index cdc01c49c9..210b25d147 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1119,21 +1126,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1147,21 +1210,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1240,6 +1312,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1691,14 +1851,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2652,6 +2807,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3005,6 +3171,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3633,6 +3861,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..b10fc9a241 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,48 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +104,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +123,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +528,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +579,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +644,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +724,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--wac7ysb48OaltWcw
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v24-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v24 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 256 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1666 insertions(+), 157 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 3c83fe6bab..d28512a638 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2301,7 +2301,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2311,7 +2311,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3635,7 +3635,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index cdc01c49c9..210b25d147 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1119,21 +1126,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1147,21 +1210,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1240,6 +1312,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1691,14 +1851,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2652,6 +2807,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3005,6 +3171,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3633,6 +3861,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..b10fc9a241 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,48 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +104,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +123,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +528,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +579,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +644,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +724,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--wac7ysb48OaltWcw
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v24-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v24 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 256 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1666 insertions(+), 157 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 3c83fe6bab..d28512a638 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2301,7 +2301,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2311,7 +2311,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3635,7 +3635,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index cdc01c49c9..210b25d147 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1119,21 +1126,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1147,21 +1210,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1240,6 +1312,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1691,14 +1851,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2652,6 +2807,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3005,6 +3171,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3633,6 +3861,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..b10fc9a241 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,48 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +104,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +123,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +528,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +579,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +644,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +724,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--wac7ysb48OaltWcw
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v24-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v27 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 250 +++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 +++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1650 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..5c3a84629f 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1588,6 +1667,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1685,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1713,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1780,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1802,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1825,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1864,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1932,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2027,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2116,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2148,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2162,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2176,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2266,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2412,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2436,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2834,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..5cd3af3053 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1282,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1663,14 +1821,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2977,6 +3130,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3820,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..c1e37f387c 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--o56keubw4jpehgac
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v27-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v25 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1665 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index d279842d3c..93a7584a0c 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2295,7 +2295,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2305,7 +2305,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3629,7 +3629,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index fc088d3f52..e85aeb5502 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1107,19 +1114,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1130,21 +1195,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1223,6 +1297,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1674,14 +1836,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2635,6 +2792,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -2988,6 +3156,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3616,6 +3846,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index fdc63e7dea..5b3c4ab637 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..1b6c78791e 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--lrZ03NoBR/3+SXJZ
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v25-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v28 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 218 +++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 257 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 206 +++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 31 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1648 insertions(+), 146 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..d6a3cae303 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1566,6 +1645,36 @@ ReleaseDeletionLock(const ObjectAddress *object)
AccessExclusiveLock);
}
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ObjectAddresses *addrs;
+ ListCell *lc;
+
+ if (list_length(collations) == 0)
+ return;
+
+ addrs = new_object_addresses();
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ add_exact_object_address(&referenced, addrs);
+ }
+
+ recordMultipleDependencies(myself, addrs->refs, addrs->numrefs,
+ DEPENDENCY_NORMAL, record_version);
+}
+
/*
* recordDependencyOnExpr - find expression dependencies
*
@@ -1588,6 +1697,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1715,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1743,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1810,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1832,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1855,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1894,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1962,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2057,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2178,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2192,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2206,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2296,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2442,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2466,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2864,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..e5155785b6 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,32 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /*
+ * recordDependencyOnSingleRelExpr gets rid of duplicated entries
+ */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1284,108 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+/* index_check_collation_version
+ * Raise a warning if the recorded and current collation version don't
+ * match.
+*/
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+/* index_check_collation_versions
+ * Check the collation version for all dependencies on the given object.
+ */
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+/* index_update_collation_version
+ * Return the current collation version for the given object.
+ */
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+/* index_update_collation_versions
+ * Record the current collation versions of all dependencies on the given
+ * object.
+ */
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2977,6 +3151,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3841,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..8137e50036 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,14 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +51,24 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +76,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +95,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant dependencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,7 +500,8 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
@@ -459,8 +509,7 @@ changeDependencyFor(Oid classId, Oid objectId,
* Returns the number of records updated.
*/
long
-changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+changeDependenciesOf(Oid classId, Oid oldObjectId, Oid newObjectId)
{
long count = 0;
Relation depRel;
@@ -485,13 +534,46 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
- depform->objid = newObjectId;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -594,6 +676,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..2966339498 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,10 +229,9 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
+long changeDependenciesOf(Oid classId, Oid oldObjectId,
Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--gclmslubawp3yaq7
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v28-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v24 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 256 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1666 insertions(+), 157 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 3c83fe6bab..d28512a638 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2301,7 +2301,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2311,7 +2311,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3635,7 +3635,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index cdc01c49c9..210b25d147 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1119,21 +1126,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1147,21 +1210,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1240,6 +1312,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1691,14 +1851,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2652,6 +2807,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3005,6 +3171,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3633,6 +3861,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..b10fc9a241 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,48 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +104,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +123,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +528,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +579,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +644,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +724,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--wac7ysb48OaltWcw
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v24-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v27 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 250 +++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 +++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1650 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..5c3a84629f 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1588,6 +1667,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1685,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1713,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1780,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1802,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1825,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1864,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1932,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2027,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2116,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2148,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2162,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2176,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2266,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2412,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2436,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2834,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..5cd3af3053 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1282,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1663,14 +1821,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2977,6 +3130,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3820,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..c1e37f387c 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--o56keubw4jpehgac
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v27-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v28 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 218 +++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 257 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 206 +++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 31 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1648 insertions(+), 146 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..d6a3cae303 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1566,6 +1645,36 @@ ReleaseDeletionLock(const ObjectAddress *object)
AccessExclusiveLock);
}
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ObjectAddresses *addrs;
+ ListCell *lc;
+
+ if (list_length(collations) == 0)
+ return;
+
+ addrs = new_object_addresses();
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ add_exact_object_address(&referenced, addrs);
+ }
+
+ recordMultipleDependencies(myself, addrs->refs, addrs->numrefs,
+ DEPENDENCY_NORMAL, record_version);
+}
+
/*
* recordDependencyOnExpr - find expression dependencies
*
@@ -1588,6 +1697,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1715,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1743,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1810,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1832,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1855,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1894,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1962,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2057,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2178,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2192,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2206,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2296,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2442,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2466,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2864,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..e5155785b6 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,32 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /*
+ * recordDependencyOnSingleRelExpr gets rid of duplicated entries
+ */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1284,108 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+/* index_check_collation_version
+ * Raise a warning if the recorded and current collation version don't
+ * match.
+*/
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+/* index_check_collation_versions
+ * Check the collation version for all dependencies on the given object.
+ */
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+/* index_update_collation_version
+ * Return the current collation version for the given object.
+ */
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+/* index_update_collation_versions
+ * Record the current collation versions of all dependencies on the given
+ * object.
+ */
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2977,6 +3151,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3841,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..8137e50036 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,14 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +51,24 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +76,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +95,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant dependencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,7 +500,8 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
@@ -459,8 +509,7 @@ changeDependencyFor(Oid classId, Oid objectId,
* Returns the number of records updated.
*/
long
-changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+changeDependenciesOf(Oid classId, Oid oldObjectId, Oid newObjectId)
{
long count = 0;
Relation depRel;
@@ -485,13 +534,46 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
- depform->objid = newObjectId;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -594,6 +676,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..2966339498 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,10 +229,9 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
+long changeDependenciesOf(Oid classId, Oid oldObjectId,
Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--gclmslubawp3yaq7
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v28-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v24 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 256 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1666 insertions(+), 157 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 3c83fe6bab..d28512a638 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2301,7 +2301,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2311,7 +2311,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3635,7 +3635,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index cdc01c49c9..210b25d147 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1119,21 +1126,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1147,21 +1210,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1240,6 +1312,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1691,14 +1851,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2652,6 +2807,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3005,6 +3171,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3633,6 +3861,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..b10fc9a241 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,48 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +104,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +123,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +528,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +579,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +644,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +724,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--wac7ysb48OaltWcw
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v24-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v17 4/7] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 2 +-
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 194 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 132 +++++++++++-
src/backend/catalog/pg_type.c | 69 +++++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/indexcmds.c | 11 +
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 ++++-
src/backend/utils/cache/relcache.c | 2 +
src/include/catalog/dependency.h | 19 +-
src/include/catalog/index.h | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++++
.../regress/expected/collate.icu.utf8.out | 157 ++++++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++++
26 files changed, 1048 insertions(+), 68 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index b84502295e..f919561923 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21123,7 +21123,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system.
+ operating system. An empty string is returned if the version is unknown.
</para>
<para>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index c54a7c420d..7209a606fa 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 281f5b7c28..ddcfb7a856 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -78,6 +78,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -138,6 +139,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -438,6 +442,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1591,6 +1670,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1605,8 +1688,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1633,12 +1716,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1694,8 +1783,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1716,8 +1805,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1739,8 +1828,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1773,6 +1867,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1798,10 +1935,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1891,7 +2030,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1979,7 +2119,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2010,7 +2151,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2023,7 +2165,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2036,7 +2179,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2125,7 +2269,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2270,7 +2415,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2292,7 +2439,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2688,8 +2837,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9d9e915979..b756af35d0 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2304,7 +2304,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2314,7 +2314,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3638,7 +3638,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 76fd938ce3..11a998d3d4 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -52,6 +52,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -74,6 +75,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -117,6 +119,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1025,6 +1028,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1115,21 +1122,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1143,21 +1206,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1229,6 +1301,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2635,6 +2795,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3623,6 +3794,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 84a0b99311..a17bbe4442 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,12 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +49,48 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +98,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +117,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -545,6 +616,55 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index cd56714968..2b62e6f47c 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 4e8263af4b..8f42da2444 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3323,6 +3323,17 @@ ReindexRelationConcurrently(Oid relationOid, int options)
/* Start a new transaction to finish process properly */
StartTransactionCommand();
+ /*
+ * Record the current versions of all depended-on collations, for all new
+ * indexes.
+ */
+ foreach(lc, newIndexIds)
+ {
+ Oid indOid = lfirst_oid(lc);
+
+ index_update_collation_versions(indOid);
+ }
+
/* Log what we did */
if (options & REINDEXOPT_VERBOSE)
{
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d625d17bf4..8f92601865 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -634,6 +636,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index d82fc5ab8b..28ef63d6ab 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 60dab33fcb..fc3a41221f 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1473,7 +1478,7 @@ pg_newlocale_from_collation(Oid collid)
* NULL (if it doesn't support versions). It must not return NULL for some
* collcollate and not NULL for others.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1511,6 +1516,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index ff70326474..bdf50ffe89 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/pg_am.h"
@@ -5623,6 +5624,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 62e86a19b6..e39e362524 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -156,7 +156,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -176,17 +177,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index a2890c1314..0899a203c0 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +133,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+void index_update_collation_versions(Oid relid);
+
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
char relpersistence, int options);
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 97890946c5..3eb5ec4fcd 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -350,6 +350,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 44ed04dd3f..82005e9eba 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index ae95bb38a6..94b4daf4d6 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--FCuugMFkClbJLl1L
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v17-0005-Preserve-index-dependencies-on-collation-during-.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v15 4/7] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 2 +-
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 189 +++++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 191 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 136 +++++++++++--
src/backend/catalog/pg_type.c | 69 +++++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 31 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 ++++-
src/backend/utils/cache/relcache.c | 2 +
src/include/catalog/dependency.h | 22 +-
src/include/catalog/index.h | 2 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++++
.../regress/expected/collate.icu.utf8.out | 130 ++++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 83 ++++++++
25 files changed, 974 insertions(+), 83 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index b2d991ac7f..8414f2bfd9 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21115,7 +21115,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system.
+ operating system. An empty string is returned if the version is unknown.
</para>
<para>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index c54a7c420d..7209a606fa 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 660033b9c1..c0e36ef4cd 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -78,6 +78,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -138,6 +139,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -438,6 +442,80 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version, *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1591,6 +1669,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1603,9 +1685,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1632,12 +1715,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,9 +1780,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, NULL,
+ self_addrs->refs,
self_addrs->numrefs,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1712,9 +1802,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1827,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1866,46 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /* Record collations from the type itself, or underlying in case of
+ * complex type. Note that if the direct parent is a CollateExpr
+ * node, there's no need to record the type underlying collation if
+ * any. A dependency already exists for the owning relation, and a
+ * change in the collation sort order wouldn't cause any harm as
+ * the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1794,11 +1930,13 @@ find_expr_references_walker(Node *node,
/*
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
- * a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * a simple constant. However we can save work in the most common case
+ * where the collation is "default", since we know that's pinned, if
+ * the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2026,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2115,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2147,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2161,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2175,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2265,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2411,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2435,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2833,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, NULL, referenced->numrefs,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9d9e915979..4546789446 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2304,7 +2304,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2314,7 +2314,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3638,7 +3638,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */,
+ false /* don't track versions */);
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 76fd938ce3..369b5d43f1 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -52,6 +52,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -74,6 +75,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -117,6 +119,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1025,6 +1028,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1115,21 +1122,75 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or not,
+ * removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1143,21 +1204,29 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1229,6 +1298,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+static void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2635,6 +2792,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3623,6 +3791,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index b7beb2884e..3f113499b2 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,16 +19,21 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -44,34 +49,47 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
- * As recordDependencyOn(), but also capture a version string so that changes
- * in the referenced object can be detected. The meaning of the version
- * string depends on the referenced object. Currently it is used for
- * detecting changes in collation versions.
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
*/
-void
-recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior)
+void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
{
- recordMultipleDependencies(depender, referenced, version, 1, behavior);
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -79,6 +97,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -97,12 +116,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries and
+ * calling CommandCounterIncrement() if the dependencies are
+ * registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -559,6 +615,54 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenceds addresses.
+ */
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index cd56714968..fe14b6bd38 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d625d17bf4..2b7de111cd 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -634,6 +636,35 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index d82fc5ab8b..f2fc427fc8 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 60dab33fcb..18a2d72f91 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1473,7 +1478,7 @@ pg_newlocale_from_collation(Oid collid)
* NULL (if it doesn't support versions). It must not return NULL for some
* collcollate and not NULL for others.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1511,6 +1516,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index ff70326474..bdf50ffe89 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/pg_am.h"
@@ -5623,6 +5624,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 7d9fc866b9..8858ceb3df 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -156,7 +156,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -176,22 +177,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
-extern void recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index a2890c1314..c619d02465 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 97890946c5..3eb5ec4fcd 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -350,6 +350,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 44ed04dd3f..3656ea94e8 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..09512c0f66 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,136 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index ae95bb38a6..94b4daf4d6 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..c8f1a620d2 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,89 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--bp/iNruPH9dso1Pn
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v15-0005-Preserve-index-dependencies-on-collation-during-.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v16 4/7] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 2 +-
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 189 +++++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 191 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 136 +++++++++++--
src/backend/catalog/pg_type.c | 69 +++++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 31 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 ++++-
src/backend/utils/cache/relcache.c | 2 +
src/include/catalog/dependency.h | 22 +-
src/include/catalog/index.h | 2 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++++
.../regress/expected/collate.icu.utf8.out | 130 ++++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 83 ++++++++
25 files changed, 974 insertions(+), 83 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 02779ab713..d28995dd5a 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21125,7 +21125,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system.
+ operating system. An empty string is returned if the version is unknown.
</para>
<para>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index c54a7c420d..7209a606fa 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 660033b9c1..c0e36ef4cd 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -78,6 +78,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -138,6 +139,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -438,6 +442,80 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version, *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1591,6 +1669,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1603,9 +1685,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1632,12 +1715,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,9 +1780,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, NULL,
+ self_addrs->refs,
self_addrs->numrefs,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1712,9 +1802,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1827,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1866,46 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /* Record collations from the type itself, or underlying in case of
+ * complex type. Note that if the direct parent is a CollateExpr
+ * node, there's no need to record the type underlying collation if
+ * any. A dependency already exists for the owning relation, and a
+ * change in the collation sort order wouldn't cause any harm as
+ * the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1794,11 +1930,13 @@ find_expr_references_walker(Node *node,
/*
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
- * a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * a simple constant. However we can save work in the most common case
+ * where the collation is "default", since we know that's pinned, if
+ * the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2026,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2115,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2147,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2161,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2175,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2265,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2411,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2435,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2833,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, NULL, referenced->numrefs,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9d9e915979..4546789446 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2304,7 +2304,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2314,7 +2314,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3638,7 +3638,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */,
+ false /* don't track versions */);
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 76fd938ce3..369b5d43f1 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -52,6 +52,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -74,6 +75,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -117,6 +119,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1025,6 +1028,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1115,21 +1122,75 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or not,
+ * removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1143,21 +1204,29 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1229,6 +1298,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+static void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2635,6 +2792,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3623,6 +3791,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index b7beb2884e..3f113499b2 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,16 +19,21 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -44,34 +49,47 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
- * As recordDependencyOn(), but also capture a version string so that changes
- * in the referenced object can be detected. The meaning of the version
- * string depends on the referenced object. Currently it is used for
- * detecting changes in collation versions.
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
*/
-void
-recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior)
+void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
{
- recordMultipleDependencies(depender, referenced, version, 1, behavior);
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -79,6 +97,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -97,12 +116,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries and
+ * calling CommandCounterIncrement() if the dependencies are
+ * registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -559,6 +615,54 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenceds addresses.
+ */
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index cd56714968..fe14b6bd38 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d625d17bf4..2b7de111cd 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -634,6 +636,35 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index d82fc5ab8b..f2fc427fc8 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 60dab33fcb..18a2d72f91 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1473,7 +1478,7 @@ pg_newlocale_from_collation(Oid collid)
* NULL (if it doesn't support versions). It must not return NULL for some
* collcollate and not NULL for others.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1511,6 +1516,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index ff70326474..bdf50ffe89 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/pg_am.h"
@@ -5623,6 +5624,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 7d9fc866b9..8858ceb3df 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -156,7 +156,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -176,22 +177,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
-extern void recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index a2890c1314..c619d02465 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 97890946c5..3eb5ec4fcd 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -350,6 +350,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 44ed04dd3f..3656ea94e8 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..09512c0f66 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,136 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index ae95bb38a6..94b4daf4d6 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..c8f1a620d2 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,89 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--sdtB3X0nJg68CQEu
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v16-0005-Preserve-index-dependencies-on-collation-during-.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v24 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 256 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1666 insertions(+), 157 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 3c83fe6bab..d28512a638 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2301,7 +2301,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2311,7 +2311,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3635,7 +3635,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index cdc01c49c9..210b25d147 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1119,21 +1126,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1147,21 +1210,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1240,6 +1312,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1691,14 +1851,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2652,6 +2807,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3005,6 +3171,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3633,6 +3861,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..b10fc9a241 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,48 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +104,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +123,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +528,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +579,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +644,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +724,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--wac7ysb48OaltWcw
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v24-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v27 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 250 +++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 +++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1650 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..5c3a84629f 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1588,6 +1667,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1685,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1713,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1780,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1802,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1825,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1864,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1932,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2027,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2116,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2148,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2162,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2176,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2266,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2412,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2436,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2834,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..5cd3af3053 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1282,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1663,14 +1821,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2977,6 +3130,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3820,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..c1e37f387c 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--o56keubw4jpehgac
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v27-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v25 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1665 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index d279842d3c..93a7584a0c 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2295,7 +2295,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2305,7 +2305,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3629,7 +3629,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index fc088d3f52..e85aeb5502 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1107,19 +1114,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1130,21 +1195,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1223,6 +1297,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1674,14 +1836,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2635,6 +2792,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -2988,6 +3156,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3616,6 +3846,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index fdc63e7dea..5b3c4ab637 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..1b6c78791e 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--lrZ03NoBR/3+SXJZ
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v25-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v28 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 218 +++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 257 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 206 +++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 31 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1648 insertions(+), 146 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..d6a3cae303 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1566,6 +1645,36 @@ ReleaseDeletionLock(const ObjectAddress *object)
AccessExclusiveLock);
}
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ObjectAddresses *addrs;
+ ListCell *lc;
+
+ if (list_length(collations) == 0)
+ return;
+
+ addrs = new_object_addresses();
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ add_exact_object_address(&referenced, addrs);
+ }
+
+ recordMultipleDependencies(myself, addrs->refs, addrs->numrefs,
+ DEPENDENCY_NORMAL, record_version);
+}
+
/*
* recordDependencyOnExpr - find expression dependencies
*
@@ -1588,6 +1697,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1715,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1743,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1810,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1832,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1855,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1894,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1962,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2057,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2178,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2192,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2206,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2296,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2442,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2466,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2864,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..e5155785b6 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,32 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /*
+ * recordDependencyOnSingleRelExpr gets rid of duplicated entries
+ */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1284,108 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+/* index_check_collation_version
+ * Raise a warning if the recorded and current collation version don't
+ * match.
+*/
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+/* index_check_collation_versions
+ * Check the collation version for all dependencies on the given object.
+ */
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+/* index_update_collation_version
+ * Return the current collation version for the given object.
+ */
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+/* index_update_collation_versions
+ * Record the current collation versions of all dependencies on the given
+ * object.
+ */
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2977,6 +3151,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3841,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..8137e50036 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,14 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +51,24 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +76,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +95,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant dependencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,7 +500,8 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
@@ -459,8 +509,7 @@ changeDependencyFor(Oid classId, Oid objectId,
* Returns the number of records updated.
*/
long
-changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+changeDependenciesOf(Oid classId, Oid oldObjectId, Oid newObjectId)
{
long count = 0;
Relation depRel;
@@ -485,13 +534,46 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
- depform->objid = newObjectId;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -594,6 +676,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..2966339498 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,10 +229,9 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
+long changeDependenciesOf(Oid classId, Oid oldObjectId,
Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--gclmslubawp3yaq7
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v28-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v25 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1665 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index d279842d3c..93a7584a0c 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2295,7 +2295,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2305,7 +2305,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3629,7 +3629,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index fc088d3f52..e85aeb5502 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1107,19 +1114,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1130,21 +1195,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1223,6 +1297,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1674,14 +1836,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2635,6 +2792,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -2988,6 +3156,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3616,6 +3846,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index fdc63e7dea..5b3c4ab637 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..1b6c78791e 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--lrZ03NoBR/3+SXJZ
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v25-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v24 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 256 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1666 insertions(+), 157 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 3c83fe6bab..d28512a638 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2301,7 +2301,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2311,7 +2311,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3635,7 +3635,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index cdc01c49c9..210b25d147 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1119,21 +1126,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1147,21 +1210,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1240,6 +1312,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1691,14 +1851,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2652,6 +2807,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3005,6 +3171,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3633,6 +3861,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..b10fc9a241 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,48 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +104,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +123,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +528,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +579,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +644,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +724,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--wac7ysb48OaltWcw
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v24-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v27 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 250 +++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 +++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1650 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..5c3a84629f 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1588,6 +1667,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1685,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1713,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1780,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1802,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1825,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1864,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1932,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2027,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2116,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2148,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2162,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2176,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2266,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2412,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2436,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2834,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..5cd3af3053 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1282,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1663,14 +1821,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2977,6 +3130,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3820,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..c1e37f387c 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--o56keubw4jpehgac
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v27-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH 4/6] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. That version is checked against current
version whenever we call get_relation_info for an index or open the parent
table during non-full VACUUM or ANALYZE. Warn that the index may be corrupted
if the versions don't match.
A new flag is added in RelationData to specify that the check has already beed
done to avoid checking and reporting the message multiple time in a backend
lifetime.
Author: Thomas Munro, Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 2 +-
src/backend/catalog/dependency.c | 189 +++++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 191 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 136 +++++++++++--
src/backend/catalog/pg_type.c | 69 +++++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 31 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 ++++-
src/backend/utils/cache/relcache.c | 2 +
src/include/catalog/dependency.h | 22 +-
src/include/catalog/index.h | 2 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++++
.../regress/expected/collate.icu.utf8.out | 130 ++++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 83 ++++++++
24 files changed, 968 insertions(+), 83 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8fa131d60d..f1ef11d60d 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21104,7 +21104,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system.
+ operating system. An empty string is returned if the version is unknown.
</para>
<para>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 78c31baa34..93f57cd633 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -77,6 +77,7 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -137,6 +138,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -437,6 +441,80 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version, *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1590,6 +1668,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,9 +1684,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1631,12 +1714,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1690,9 +1779,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, NULL,
+ self_addrs->refs,
self_addrs->numrefs,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,9 +1801,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1735,8 +1826,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1769,6 +1865,46 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /* Record collations from the type itself, or underlying in case of
+ * complex type. Note that if the direct parent is a CollateExpr
+ * node, there's no need to record the type underlying collation if
+ * any. A dependency already exists for the owning relation, and a
+ * change in the collation sort order wouldn't cause any harm as
+ * the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,11 +1929,13 @@ find_expr_references_walker(Node *node,
/*
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
- * a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * a simple constant. However we can save work in the most common case
+ * where the collation is "default", since we know that's pinned, if
+ * the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1887,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1975,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2006,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2019,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2032,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2121,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2266,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2288,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2684,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, NULL, referenced->numrefs,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index e31478bf91..46e9d74a97 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2304,7 +2304,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2314,7 +2314,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3638,7 +3638,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */,
+ false /* don't track versions */);
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 8880586c37..c6c51e0a48 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -52,6 +52,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -74,6 +75,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -117,6 +119,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1025,6 +1028,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1115,21 +1122,75 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or not,
+ * removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1143,21 +1204,29 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1229,6 +1298,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+static void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2628,6 +2785,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3605,6 +3773,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 3d2b1cc911..0eaefbd032 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 7fdbdf0ae8..7e087fefc4 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,16 +19,21 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -44,34 +49,47 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
- * As recordDependencyOn(), but also capture a version string so that changes
- * in the referenced object can be detected. The meaning of the version
- * string depends on the referenced object. Currently it is used for
- * detecting changes in collation versions.
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
*/
-void
-recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior)
+void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
{
- recordMultipleDependencies(depender, referenced, version, 1, behavior);
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -79,6 +97,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -97,12 +116,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries and
+ * calling CommandCounterIncrement() if the dependencies are
+ * registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -559,6 +615,54 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenceds addresses.
+ */
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 8d7572da51..6e91518694 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -511,6 +512,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 493aa21a14..05709fab77 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -272,28 +272,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d625d17bf4..2b7de111cd 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -634,6 +636,35 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index d82fc5ab8b..f2fc427fc8 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 597c1241f9..7dacceefeb 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -142,6 +144,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1463,7 +1468,7 @@ pg_newlocale_from_collation(Oid collid)
* NULL (if it doesn't support versions). It must not return NULL for some
* collcollate and not NULL for others.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1501,6 +1506,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index ff70326474..bdf50ffe89 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/pg_am.h"
@@ -5623,6 +5624,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 77cf0612ed..8750bfc36f 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -156,7 +156,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -176,22 +177,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
-extern void recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index a2890c1314..c619d02465 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index e1a5ab3df3..2bf6f868a5 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -335,6 +335,8 @@ extern void GenerateTypeDependencies(Oid typeObjectId,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 44ed04dd3f..3656ea94e8 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..09512c0f66 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,136 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 6ddf3a63c3..92dcda5bdc 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..c8f1a620d2 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,89 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.25.1
--6jmy5gypzsv7ulgm
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
filename*0=0005-Preserve-index-dependencies-on-collation-during-pg_u-v11;
filename*1=".patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v30 4/6] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 +
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 219 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 307 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 262 ++++++++++++++-
src/backend/catalog/pg_type.c | 69 ++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 ++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 ++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 31 +-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 187 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 126 +++++++
37 files changed, 1803 insertions(+), 148 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 1f33f99040..86f2e16533 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25462,7 +25462,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index b59c5697a3..527c847e27 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index fa43e3a972..ef047a8945 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..c93fdbd7d5 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1566,6 +1645,37 @@ ReleaseDeletionLock(const ObjectAddress *object)
AccessExclusiveLock);
}
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ObjectAddresses *addrs;
+ ListCell *lc;
+
+ if (list_length(collations) == 0)
+ return;
+
+ addrs = new_object_addresses();
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ add_exact_object_address(&referenced, addrs);
+ }
+
+ eliminate_duplicate_dependencies(addrs);
+ recordMultipleDependencies(myself, addrs->refs, addrs->numrefs,
+ DEPENDENCY_NORMAL, record_version);
+}
+
/*
* recordDependencyOnExpr - find expression dependencies
*
@@ -1588,6 +1698,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1716,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1744,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1811,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1833,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1856,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1895,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1963,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2058,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2147,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2179,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2193,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2207,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2297,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2443,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2467,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2865,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 67144aa3c9..fc5140daa6 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2336,7 +2336,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2346,7 +2346,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3665,7 +3665,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 0974f3e23a..4c8a221e0d 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/rel.h"
@@ -1020,6 +1022,11 @@ index_create(Relation heapRelation,
ObjectAddress myself,
referenced;
ObjectAddresses *addrs;
+ ListCell *lc;
+ List *colls = NIL,
+ *colls_pattern_ops = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1106,20 +1113,122 @@ index_create(Relation heapRelation,
/* placeholder for normal dependencies */
addrs = new_object_addresses();
- /* Store dependency on collations */
-
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- add_exact_object_address(&referenced, addrs);
+ Oid opclass = classObjectId[i];
+
+ /*
+ * The *_pattern_ops opclasses are special, as they're known
+ * to not rely on the collation sort order.
+ */
+ if (opclass == TEXT_BTREE_PATTERN_OPS_OID ||
+ opclass == VARCHAR_BTREE_PATTERN_OPS_OID ||
+ opclass == BPCHAR_BTREE_PATTERN_OPS_OID)
+ colls_pattern_ops = lappend_oid(colls_pattern_ops, colloid);
+ else
+ colls = lappend_oid(colls, colloid);
+ }
+ /*
+ * Required for e.g. custom types having multiple underlying
+ * collations. Note that none of the *_pattern_ops can be used in
+ * such constructs.
+ */
+ else
+ {
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /* Check if any collation exist for both a *_pattern_ops opclass and
+ * another one. In that case, we have to create a single dependency
+ * with version tracked.
+ */
+ if (colls_pattern_ops != NIL && colls != NIL)
+ colls_pattern_ops = list_difference_oid(colls_pattern_ops, colls);
+
+ /*
+ * Record the dependencies for collation declares with any of the
+ * *_pattern_ops opclass, without version tracking.
+ */
+ if (colls_pattern_ops != NIL)
+ {
+ recordDependencyOnCollations(&myself, colls_pattern_ops, false);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * Then split the dependencies for collations that were not declared
+ * with any of the *_pattern_ops opclass on whether they're
+ * deterministic or not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = lappend_oid(determ_colls, c);
+ else
+ nondeterm_colls = lappend_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic collation, we always track the dependency on the
+ * collation but only track the version if the AM relies on a stable
+ * ordering.
+ */
+ if (determ_colls != NIL)
+ {
+ IndexAmRoutine *routine;
+ bool track_version;
+
+ routine = GetIndexAmRoutineByAmId(accessMethodObjectId, false);
+ track_version = !routine->amnostablecollorder;
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls != NIL)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1133,21 +1242,32 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /*
+ * recordDependencyOnSingleRelExpr gets rid of duplicated entries
+ */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1226,6 +1346,108 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+/* index_check_collation_version
+ * Raise a warning if the recorded and current collation version don't
+ * match.
+*/
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+/* index_check_collation_versions
+ * Check the collation version for all dependencies on the given object.
+ */
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+/* index_update_collation_version
+ * Return the current collation version for the given object.
+ */
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+/* index_update_collation_versions
+ * Record the current collation versions of all dependencies on the given
+ * object.
+ */
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -3000,6 +3222,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3635,6 +3919,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 0d70cb0c3c..93774c9d21 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -362,7 +362,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 09c30b13e8..89c866cb04 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,15 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ char *version);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +52,24 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -66,6 +78,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
max_slots,
slot_init_count,
slot_stored_count;
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -96,12 +109,55 @@ recordMultipleDependencies(const ObjectAddress *depender,
slot_init_count = 0;
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant dependencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ * However, we may have already recorded a dependency on a
+ * collation that didn't required to track the version (if any
+ * of the *_pattern_ops opclasses was used). If we now see a
+ * dependency on the same collation that requires to track the
+ * version, we need to update it with the current version,
+ * which is done by dependencyExists().
+ */
+ version = get_collation_version_for_oid(referenced->objectId);
+ if (dependencyExists(depender, referenced, version))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (isObjectPinned(referenced, dependDesc))
+ if (!ignore_systempin && isObjectPinned(referenced, dependDesc))
continue;
if (slot_init_count < max_slots)
@@ -493,7 +549,8 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
@@ -501,8 +558,7 @@ changeDependencyFor(Oid classId, Oid objectId,
* Returns the number of records updated.
*/
long
-changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+changeDependenciesOf(Oid classId, Oid oldObjectId, Oid newObjectId)
{
long count = 0;
Relation depRel;
@@ -526,14 +582,47 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
- Form_pg_depend depform;
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -636,6 +725,151 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ *
+ * If a record is found, also update the stored dependency version if it wasn't
+ * previously tracked.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced,
+ char *version)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ char *cur_version = NULL;
+ Datum depversion;
+ bool isnull;
+
+ ret = true;
+
+ /*
+ * We only need the retrieve the current version if a new version
+ * was passed.
+ */
+ if (version)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+ }
+
+ /* Track version if it wasn't yet and we have a new version. */
+ if (version && !cur_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ else
+ {
+ Assert(strcmp(cur_version, version) == 0);
+ }
+
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 0b04dff773..f2af6212e5 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index ddeec870d8..e1016609dc 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index f9d0d67aa7..3a20dcdee5 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 9061af81a3..153adc157d 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5934,6 +5935,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2174d66c1f..cc5accc84c 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7024,7 +7032,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7060,7 +7070,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7085,7 +7150,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7124,7 +7191,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7159,7 +7228,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7190,7 +7261,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7224,7 +7297,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7264,6 +7339,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7289,6 +7366,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16353,7 +16432,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16420,6 +16500,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16448,6 +16532,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18434,6 +18533,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index e0b42e8391..623814d1c5 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if a partition, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..2966339498 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,10 +229,9 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
+long changeDependenciesOf(Oid classId, Oid oldObjectId,
Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index f48f5fb4d9..daf29aa6c5 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10367,6 +10367,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1488bffa2b..8c54e12268 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..68fe86cf89 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,193 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ val text COLLATE "fr-x-icu",
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx00_val ON collate_test(val);
+-- shouldn't get duplicated dependencies
+CREATE INDEX icuidx00_val_val ON collate_test(val, val);
+-- shouldn't track version
+CREATE INDEX icuidx00_val_pattern ON collate_test(val text_pattern_ops);
+-- should have single dependency, no version tracked
+CREATE INDEX icuidx00_val_pattern_val_pattern ON collate_test(val text_pattern_ops, val text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_val ON collate_test(val text_pattern_ops, val);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_val_pattern ON collate_test(val, val text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_where ON collate_test(val text_pattern_ops) WHERE val >= val;
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_where ON collate_test(val) WHERE val >= val;
+-- should ideally have single dependency, no version tracked, but expression walker will find a dependency on the collation and will ask to track the version
+CREATE INDEX icuidx00_val_pattern_expr_pattern ON collate_test(val varchar_pattern_ops, (val || val) text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_expr ON collate_test(val varchar_pattern_ops, (val || val));
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' COLLATE custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' COLLATE custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' COLLATE "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+-----------------------------------+------------+---------------------
+ icuidx00_val | "fr-x-icu" | up to date
+ icuidx00_val_val | "fr-x-icu" | up to date
+ icuidx00_val_pattern | "fr-x-icu" | version not tracked
+ icuidx00_val_pattern_val_pattern | "fr-x-icu" | version not tracked
+ icuidx00_val_pattern_val | "fr-x-icu" | up to date
+ icuidx00_val_val_pattern | "fr-x-icu" | up to date
+ icuidx00_val_pattern_where | "fr-x-icu" | up to date
+ icuidx00_val_where | "fr-x-icu" | up to date
+ icuidx00_val_pattern_expr_pattern | "fr-x-icu" | up to date
+ icuidx00_val_pattern_expr | "fr-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(59 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 6ace7662ee..6cb7786e13 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,14 +2065,16 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(8 rows)
+(10 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2092,14 +2094,16 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(8 rows)
+(10 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..fbe220df41 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,132 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ val text COLLATE "fr-x-icu",
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx00_val ON collate_test(val);
+-- shouldn't get duplicated dependencies
+CREATE INDEX icuidx00_val_val ON collate_test(val, val);
+-- shouldn't track version
+CREATE INDEX icuidx00_val_pattern ON collate_test(val text_pattern_ops);
+-- should have single dependency, no version tracked
+CREATE INDEX icuidx00_val_pattern_val_pattern ON collate_test(val text_pattern_ops, val text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_val ON collate_test(val text_pattern_ops, val);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_val_pattern ON collate_test(val, val text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_where ON collate_test(val text_pattern_ops) WHERE val >= val;
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_where ON collate_test(val) WHERE val >= val;
+-- should ideally have single dependency, no version tracked, but expression walker will find a dependency on the collation and will ask to track the version
+CREATE INDEX icuidx00_val_pattern_expr_pattern ON collate_test(val varchar_pattern_ops, (val || val) text_pattern_ops);
+-- should have single dependency, with version tracked
+CREATE INDEX icuidx00_val_pattern_expr ON collate_test(val varchar_pattern_ops, (val || val));
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' COLLATE custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' COLLATE custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' COLLATE "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--vy36zrg552ym33jw
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v30-0005-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v27 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 250 +++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 +++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1650 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..5c3a84629f 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1588,6 +1667,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1685,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1713,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1780,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1802,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1825,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1864,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1932,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2027,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2116,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2148,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2162,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2176,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2266,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2412,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2436,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2834,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..5cd3af3053 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1282,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1663,14 +1821,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2977,6 +3130,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3820,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..c1e37f387c 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--o56keubw4jpehgac
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v27-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v28 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 218 +++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 257 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 206 +++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 31 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1648 insertions(+), 146 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..d6a3cae303 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1566,6 +1645,36 @@ ReleaseDeletionLock(const ObjectAddress *object)
AccessExclusiveLock);
}
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ObjectAddresses *addrs;
+ ListCell *lc;
+
+ if (list_length(collations) == 0)
+ return;
+
+ addrs = new_object_addresses();
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ add_exact_object_address(&referenced, addrs);
+ }
+
+ recordMultipleDependencies(myself, addrs->refs, addrs->numrefs,
+ DEPENDENCY_NORMAL, record_version);
+}
+
/*
* recordDependencyOnExpr - find expression dependencies
*
@@ -1588,6 +1697,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1715,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1743,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1810,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1832,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1855,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1894,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1962,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2057,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2178,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2192,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2206,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2296,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2442,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2466,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2864,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..e5155785b6 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,32 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /*
+ * recordDependencyOnSingleRelExpr gets rid of duplicated entries
+ */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1284,108 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+/* index_check_collation_version
+ * Raise a warning if the recorded and current collation version don't
+ * match.
+*/
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+/* index_check_collation_versions
+ * Check the collation version for all dependencies on the given object.
+ */
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+/* index_update_collation_version
+ * Return the current collation version for the given object.
+ */
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+/* index_update_collation_versions
+ * Record the current collation versions of all dependencies on the given
+ * object.
+ */
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2977,6 +3151,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3841,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..8137e50036 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,14 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +51,24 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +76,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +95,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant dependencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,7 +500,8 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
@@ -459,8 +509,7 @@ changeDependencyFor(Oid classId, Oid objectId,
* Returns the number of records updated.
*/
long
-changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+changeDependenciesOf(Oid classId, Oid oldObjectId, Oid newObjectId)
{
long count = 0;
Relation depRel;
@@ -485,13 +534,46 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
- depform->objid = newObjectId;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -594,6 +676,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..2966339498 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,10 +229,9 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
+long changeDependenciesOf(Oid classId, Oid oldObjectId,
Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--gclmslubawp3yaq7
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v28-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v25 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1665 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index d279842d3c..93a7584a0c 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2295,7 +2295,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2305,7 +2305,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3629,7 +3629,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index fc088d3f52..e85aeb5502 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1107,19 +1114,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1130,21 +1195,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1223,6 +1297,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1674,14 +1836,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2635,6 +2792,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -2988,6 +3156,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3616,6 +3846,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index fdc63e7dea..5b3c4ab637 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..1b6c78791e 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--lrZ03NoBR/3+SXJZ
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v25-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v27 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 250 +++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 +++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1650 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..5c3a84629f 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1588,6 +1667,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1685,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1713,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1780,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1802,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1825,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1864,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1932,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2027,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2116,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2148,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2162,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2176,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2266,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2412,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2436,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2834,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..5cd3af3053 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1282,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1663,14 +1821,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2977,6 +3130,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3820,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..c1e37f387c 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--o56keubw4jpehgac
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v27-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v28 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 218 +++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 257 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 206 +++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 31 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1648 insertions(+), 146 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..d6a3cae303 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1566,6 +1645,36 @@ ReleaseDeletionLock(const ObjectAddress *object)
AccessExclusiveLock);
}
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ObjectAddresses *addrs;
+ ListCell *lc;
+
+ if (list_length(collations) == 0)
+ return;
+
+ addrs = new_object_addresses();
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ add_exact_object_address(&referenced, addrs);
+ }
+
+ recordMultipleDependencies(myself, addrs->refs, addrs->numrefs,
+ DEPENDENCY_NORMAL, record_version);
+}
+
/*
* recordDependencyOnExpr - find expression dependencies
*
@@ -1588,6 +1697,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1715,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1743,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1810,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1832,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1855,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1894,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1962,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2057,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2178,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2192,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2206,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2296,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2442,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2466,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2864,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..e5155785b6 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,32 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /*
+ * recordDependencyOnSingleRelExpr gets rid of duplicated entries
+ */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1284,108 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+/* index_check_collation_version
+ * Raise a warning if the recorded and current collation version don't
+ * match.
+*/
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+/* index_check_collation_versions
+ * Check the collation version for all dependencies on the given object.
+ */
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+/* index_update_collation_version
+ * Return the current collation version for the given object.
+ */
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+/* index_update_collation_versions
+ * Record the current collation versions of all dependencies on the given
+ * object.
+ */
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2977,6 +3151,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3841,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..8137e50036 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,14 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +51,24 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +76,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +95,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant dependencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,7 +500,8 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
@@ -459,8 +509,7 @@ changeDependencyFor(Oid classId, Oid objectId,
* Returns the number of records updated.
*/
long
-changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+changeDependenciesOf(Oid classId, Oid oldObjectId, Oid newObjectId)
{
long count = 0;
Relation depRel;
@@ -485,13 +534,46 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
- depform->objid = newObjectId;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -594,6 +676,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..2966339498 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,10 +229,9 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
+long changeDependenciesOf(Oid classId, Oid oldObjectId,
Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--gclmslubawp3yaq7
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v28-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v28 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 218 +++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 257 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 206 +++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 31 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1648 insertions(+), 146 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..d6a3cae303 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1566,6 +1645,36 @@ ReleaseDeletionLock(const ObjectAddress *object)
AccessExclusiveLock);
}
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ObjectAddresses *addrs;
+ ListCell *lc;
+
+ if (list_length(collations) == 0)
+ return;
+
+ addrs = new_object_addresses();
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ add_exact_object_address(&referenced, addrs);
+ }
+
+ recordMultipleDependencies(myself, addrs->refs, addrs->numrefs,
+ DEPENDENCY_NORMAL, record_version);
+}
+
/*
* recordDependencyOnExpr - find expression dependencies
*
@@ -1588,6 +1697,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1715,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1743,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1810,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1832,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1855,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1894,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1962,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2057,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2178,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2192,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2206,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2296,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2442,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2466,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2864,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..e5155785b6 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,32 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /*
+ * recordDependencyOnSingleRelExpr gets rid of duplicated entries
+ */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1284,108 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+/* index_check_collation_version
+ * Raise a warning if the recorded and current collation version don't
+ * match.
+*/
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+/* index_check_collation_versions
+ * Check the collation version for all dependencies on the given object.
+ */
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+/* index_update_collation_version
+ * Return the current collation version for the given object.
+ */
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+/* index_update_collation_versions
+ * Record the current collation versions of all dependencies on the given
+ * object.
+ */
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2977,6 +3151,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3841,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..8137e50036 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,14 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +51,24 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +76,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +95,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant dependencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,7 +500,8 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
@@ -459,8 +509,7 @@ changeDependencyFor(Oid classId, Oid objectId,
* Returns the number of records updated.
*/
long
-changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+changeDependenciesOf(Oid classId, Oid oldObjectId, Oid newObjectId)
{
long count = 0;
Relation depRel;
@@ -485,13 +534,46 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
- depform->objid = newObjectId;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -594,6 +676,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..2966339498 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,10 +229,9 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
+long changeDependenciesOf(Oid classId, Oid oldObjectId,
Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--gclmslubawp3yaq7
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v28-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v27 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 250 +++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 +++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1650 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..5c3a84629f 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1588,6 +1667,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1685,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1713,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1780,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1802,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1825,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1864,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1932,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2027,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2116,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2148,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2162,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2176,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2266,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2412,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2436,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2834,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..5cd3af3053 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1282,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1663,14 +1821,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2977,6 +3130,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3820,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..c1e37f387c 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--o56keubw4jpehgac
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v27-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v28 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 218 +++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 257 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 206 +++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 31 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1648 insertions(+), 146 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..d6a3cae303 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1566,6 +1645,36 @@ ReleaseDeletionLock(const ObjectAddress *object)
AccessExclusiveLock);
}
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ObjectAddresses *addrs;
+ ListCell *lc;
+
+ if (list_length(collations) == 0)
+ return;
+
+ addrs = new_object_addresses();
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ add_exact_object_address(&referenced, addrs);
+ }
+
+ recordMultipleDependencies(myself, addrs->refs, addrs->numrefs,
+ DEPENDENCY_NORMAL, record_version);
+}
+
/*
* recordDependencyOnExpr - find expression dependencies
*
@@ -1588,6 +1697,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1715,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1743,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1810,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1832,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1855,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1894,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1962,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2057,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2178,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2192,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2206,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2296,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2442,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2466,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2864,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..e5155785b6 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,32 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /*
+ * recordDependencyOnSingleRelExpr gets rid of duplicated entries
+ */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1284,108 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+/* index_check_collation_version
+ * Raise a warning if the recorded and current collation version don't
+ * match.
+*/
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+/* index_check_collation_versions
+ * Check the collation version for all dependencies on the given object.
+ */
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+/* index_update_collation_version
+ * Return the current collation version for the given object.
+ */
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+/* index_update_collation_versions
+ * Record the current collation versions of all dependencies on the given
+ * object.
+ */
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2977,6 +3151,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3841,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..8137e50036 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,14 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +51,24 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +76,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +95,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant dependencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,7 +500,8 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
@@ -459,8 +509,7 @@ changeDependencyFor(Oid classId, Oid objectId,
* Returns the number of records updated.
*/
long
-changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+changeDependenciesOf(Oid classId, Oid oldObjectId, Oid newObjectId)
{
long count = 0;
Relation depRel;
@@ -485,13 +534,46 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
- depform->objid = newObjectId;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -594,6 +676,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..2966339498 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,10 +229,9 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
+long changeDependenciesOf(Oid classId, Oid oldObjectId,
Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--gclmslubawp3yaq7
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v28-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v28 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 218 +++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 257 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 206 +++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 31 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1648 insertions(+), 146 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..d6a3cae303 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1566,6 +1645,36 @@ ReleaseDeletionLock(const ObjectAddress *object)
AccessExclusiveLock);
}
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ObjectAddresses *addrs;
+ ListCell *lc;
+
+ if (list_length(collations) == 0)
+ return;
+
+ addrs = new_object_addresses();
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ add_exact_object_address(&referenced, addrs);
+ }
+
+ recordMultipleDependencies(myself, addrs->refs, addrs->numrefs,
+ DEPENDENCY_NORMAL, record_version);
+}
+
/*
* recordDependencyOnExpr - find expression dependencies
*
@@ -1588,6 +1697,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1715,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1743,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1810,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1832,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1855,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1894,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1962,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2057,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2178,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2192,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2206,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2296,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2442,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2466,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2864,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..e5155785b6 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,32 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /*
+ * recordDependencyOnSingleRelExpr gets rid of duplicated entries
+ */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1284,108 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+/* index_check_collation_version
+ * Raise a warning if the recorded and current collation version don't
+ * match.
+*/
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+/* index_check_collation_versions
+ * Check the collation version for all dependencies on the given object.
+ */
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+/* index_update_collation_version
+ * Return the current collation version for the given object.
+ */
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+/* index_update_collation_versions
+ * Record the current collation versions of all dependencies on the given
+ * object.
+ */
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2977,6 +3151,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3841,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..8137e50036 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,14 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +51,24 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +76,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +95,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant dependencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,7 +500,8 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
@@ -459,8 +509,7 @@ changeDependencyFor(Oid classId, Oid objectId,
* Returns the number of records updated.
*/
long
-changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+changeDependenciesOf(Oid classId, Oid oldObjectId, Oid newObjectId)
{
long count = 0;
Relation depRel;
@@ -485,13 +534,46 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
- depform->objid = newObjectId;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -594,6 +676,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..2966339498 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,10 +229,9 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
+long changeDependenciesOf(Oid classId, Oid oldObjectId,
Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--gclmslubawp3yaq7
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v28-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v24 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 256 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1666 insertions(+), 157 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 3c83fe6bab..d28512a638 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2301,7 +2301,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2311,7 +2311,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3635,7 +3635,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index cdc01c49c9..210b25d147 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1119,21 +1126,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1147,21 +1210,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1240,6 +1312,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1691,14 +1851,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2652,6 +2807,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3005,6 +3171,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3633,6 +3861,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..b10fc9a241 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,48 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +104,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +123,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +528,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +579,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +644,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +724,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--wac7ysb48OaltWcw
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v24-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v24 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 256 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1666 insertions(+), 157 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 3c83fe6bab..d28512a638 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2301,7 +2301,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2311,7 +2311,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3635,7 +3635,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index cdc01c49c9..210b25d147 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1119,21 +1126,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1147,21 +1210,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1240,6 +1312,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1691,14 +1851,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2652,6 +2807,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3005,6 +3171,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3633,6 +3861,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..b10fc9a241 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,48 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +104,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +123,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +528,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +579,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +644,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +724,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--wac7ysb48OaltWcw
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v24-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v25 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1665 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index d279842d3c..93a7584a0c 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2295,7 +2295,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2305,7 +2305,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3629,7 +3629,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index fc088d3f52..e85aeb5502 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1107,19 +1114,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1130,21 +1195,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1223,6 +1297,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1674,14 +1836,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2635,6 +2792,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -2988,6 +3156,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3616,6 +3846,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index fdc63e7dea..5b3c4ab637 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..1b6c78791e 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--lrZ03NoBR/3+SXJZ
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v25-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH 4/6] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. That version is checked against current
version whenever we call get_relation_info for an index or open the parent
table during non-full VACUUM or ANALYZE. Warn that the index may be corrupted
if the versions don't match.
A new flag is added in RelationData to specify that the check has already beed
done to avoid checking and reporting the message multiple time in a backend
lifetime.
Author: Thomas Munro, Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/catalog/dependency.c | 189 ++++++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 139 ++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 127 ++++++++++--
src/backend/catalog/pg_type.c | 69 +++++++
src/backend/commands/vacuum.c | 31 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 38 ++++
src/backend/utils/cache/relcache.c | 2 +
src/include/catalog/dependency.h | 21 +-
src/include/catalog/index.h | 2 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 1 +
src/include/utils/rel.h | 1 +
.../regress/expected/collate.icu.utf8.out | 118 +++++++++++
...te.icu.utf8.out => collate.icu.utf8_2.out} | 118 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 76 +++++++
19 files changed, 901 insertions(+), 59 deletions(-)
copy src/test/regress/expected/{collate.icu.utf8.out => collate.icu.utf8_2.out} (89%)
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 78c31baa34..93f57cd633 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -77,6 +77,7 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -137,6 +138,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -437,6 +441,80 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version, *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1590,6 +1668,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,9 +1684,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1631,12 +1714,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1690,9 +1779,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, NULL,
+ self_addrs->refs,
self_addrs->numrefs,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,9 +1801,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1735,8 +1826,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1769,6 +1865,46 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /* Record collations from the type itself, or underlying in case of
+ * complex type. Note that if the direct parent is a CollateExpr
+ * node, there's no need to record the type underlying collation if
+ * any. A dependency already exists for the owning relation, and a
+ * change in the collation sort order wouldn't cause any harm as
+ * the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,11 +1929,13 @@ find_expr_references_walker(Node *node,
/*
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
- * a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * a simple constant. However we can save work in the most common case
+ * where the collation is "default", since we know that's pinned, if
+ * the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1887,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1975,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2006,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2019,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2032,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2121,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2266,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2288,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2684,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, NULL, referenced->numrefs,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index e31478bf91..46e9d74a97 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2304,7 +2304,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2314,7 +2314,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3638,7 +3638,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */,
+ false /* don't track versions */);
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 8880586c37..cd5c93d4ef 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -74,6 +74,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -724,6 +725,7 @@ index_create(Relation heapRelation,
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
+ List *collations = NIL;
/* constraint flags can only be set when a constraint is requested */
Assert((constr_flags == 0) ||
@@ -1115,21 +1117,44 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /*
+ * Get required distinct dependencies on collations for all index keys.
+ * Collations of directly referenced column in hash indexes can be
+ * skipped is they're deterministic.
+ */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ if ((indexInfo->ii_Am != HASH_AM_OID) ||
+ !get_collation_isdeterministic(colloid))
+ collations = list_append_unique_oid(collations, colloid);
+ }
+ else
+ {
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ collations = list_concat_unique_oid(collations,
+ GetTypeCollations(att->atttypid,
+ (indexInfo->ii_Am == HASH_AM_OID)));
}
}
+ if (collations)
+ {
+ recordDependencyOnCollations(&myself, collations);
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1143,21 +1168,29 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1229,6 +1262,89 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ if (!version)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+static void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -3605,6 +3721,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 3d2b1cc911..0eaefbd032 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 7fdbdf0ae8..709e7f5530 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,16 +19,21 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -44,22 +49,29 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
- * As recordDependencyOn(), but also capture a version string so that changes
- * in the referenced object can be detected. The meaning of the version
- * string depends on the referenced object. Currently it is used for
- * detecting changes in collation versions.
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
*/
-void
-recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior)
+void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations)
{
- recordMultipleDependencies(depender, referenced, version, 1, behavior);
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, true);
+ }
}
/*
@@ -69,9 +81,9 @@ recordDependencyOnVersion(const ObjectAddress *depender,
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -79,6 +91,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -97,12 +110,46 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation needs to be tracked */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries and
+ * calling CommandCounterIncrement() if the dependencies are
+ * registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;;
+ version = get_collation_version_for_oid(referenced->objectId);
+ }
+ }
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -559,6 +606,54 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenceds addresses.
+ */
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 8d7572da51..6e91518694 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -511,6 +512,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d625d17bf4..2b7de111cd 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -634,6 +636,35 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index d82fc5ab8b..f2fc427fc8 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 597c1241f9..123bfa6c01 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -1501,6 +1503,42 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version;
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index ff70326474..bdf50ffe89 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/pg_am.h"
@@ -5623,6 +5624,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 77cf0612ed..6de17e25e8 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -156,7 +156,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -176,22 +177,28 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
-extern void recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations);
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index a2890c1314..c619d02465 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index e1a5ab3df3..2bf6f868a5 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -335,6 +335,8 @@ extern void GenerateTypeDependencies(Oid typeObjectId,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..aa06aca90a 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -104,6 +104,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 44ed04dd3f..3656ea94e8 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..f049189897 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,124 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- version of default (and libc) collation is only tracked on glibc systems
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'no version'
+WHEN refobjversion = '' THEN 'unknown version'
+ELSE 'some version' END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+--------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | some version
+ icuidx01_t_en_fr__d_es | "es-x-icu" | some version
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | some version
+ icuidx02_d_en_fr | "en-x-icu" | some version
+ icuidx02_d_en_fr | "fr-x-icu" | some version
+ icuidx03_t_en_fr_ga | "en-x-icu" | some version
+ icuidx03_t_en_fr_ga | "fr-x-icu" | some version
+ icuidx03_t_en_fr_ga | "ga-x-icu" | some version
+ icuidx04_d_en_fr_ga | "en-x-icu" | some version
+ icuidx04_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx04_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | some version
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | some version
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | some version
+ icuidx06_d_en_fr_ga | "default" | no version
+ icuidx06_d_en_fr_ga | "en-x-icu" | some version
+ icuidx06_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx06_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx07_d_en_fr_ga | "default" | no version
+ icuidx07_d_en_fr_ga | "en-x-icu" | some version
+ icuidx07_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx07_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx08_d_en_fr_ga | "en-x-icu" | some version
+ icuidx08_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx08_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx09_d_en_fr_ga | "en-x-icu" | some version
+ icuidx09_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx09_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | some version
+ icuidx11_d_es | "default" | no version
+ icuidx11_d_es | "es-x-icu" | some version
+ icuidx12_custom | "default" | no version
+ icuidx12_custom | custom | some version
+ icuidx13_custom | "default" | no version
+ icuidx13_custom | custom | some version
+ icuidx14_myrange | "default" | no version
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | some version
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | some version
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | some version
+ icuidx16_mood | "fr-x-icu" | some version
+ icuidx17_part | "en-x-icu" | some version
+(44 rows)
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8_2.out
similarity index 89%
copy from src/test/regress/expected/collate.icu.utf8.out
copy to src/test/regress/expected/collate.icu.utf8_2.out
index 60d9263a2f..07e76c4f48 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8_2.out
@@ -1897,6 +1897,124 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- version of default (and libc) collation is only tracked on glibc systems
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'no version'
+WHEN refobjversion = '' THEN 'unknown version'
+ELSE 'some version' END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+--------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | some version
+ icuidx01_t_en_fr__d_es | "es-x-icu" | some version
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | some version
+ icuidx02_d_en_fr | "en-x-icu" | some version
+ icuidx02_d_en_fr | "fr-x-icu" | some version
+ icuidx03_t_en_fr_ga | "en-x-icu" | some version
+ icuidx03_t_en_fr_ga | "fr-x-icu" | some version
+ icuidx03_t_en_fr_ga | "ga-x-icu" | some version
+ icuidx04_d_en_fr_ga | "en-x-icu" | some version
+ icuidx04_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx04_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | some version
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | some version
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | some version
+ icuidx06_d_en_fr_ga | "default" | some version
+ icuidx06_d_en_fr_ga | "en-x-icu" | some version
+ icuidx06_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx06_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx07_d_en_fr_ga | "default" | some version
+ icuidx07_d_en_fr_ga | "en-x-icu" | some version
+ icuidx07_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx07_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx08_d_en_fr_ga | "en-x-icu" | some version
+ icuidx08_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx08_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx09_d_en_fr_ga | "en-x-icu" | some version
+ icuidx09_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx09_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | some version
+ icuidx11_d_es | "default" | some version
+ icuidx11_d_es | "es-x-icu" | some version
+ icuidx12_custom | "default" | some version
+ icuidx12_custom | custom | some version
+ icuidx13_custom | "default" | some version
+ icuidx13_custom | custom | some version
+ icuidx14_myrange | "default" | some version
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | some version
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | some version
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | some version
+ icuidx16_mood | "fr-x-icu" | some version
+ icuidx17_part | "en-x-icu" | some version
+(44 rows)
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 6ddf3a63c3..92dcda5bdc 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..920a73512f 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,82 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+
+-- version of default (and libc) collation is only tracked on glibc systems
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'no version'
+WHEN refobjversion = '' THEN 'unknown version'
+ELSE 'some version' END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--6c2NcOVqGQ03X4Wi
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename*0="0005-Preserve-index-dependencies-on-collation-during-pg_u-v8.";
filename*1=patch
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v19 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 2 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 256 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1665 insertions(+), 157 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 822c4405af..9c9ab19ec0 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21192,7 +21192,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system.
+ operating system. An empty string is returned if the version is unknown.
</para>
<para>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 997ef1e12a..4b26838ad0 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -212,6 +212,24 @@
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index c54a7c420d..7209a606fa 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 281f5b7c28..ddcfb7a856 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -78,6 +78,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -138,6 +139,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -438,6 +442,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1591,6 +1670,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1605,8 +1688,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1633,12 +1716,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1694,8 +1783,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1716,8 +1805,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1739,8 +1828,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1773,6 +1867,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1798,10 +1935,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1891,7 +2030,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1979,7 +2119,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2010,7 +2151,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2023,7 +2165,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2036,7 +2179,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2125,7 +2269,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2270,7 +2415,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2292,7 +2439,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2688,8 +2837,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 632c058b80..0e6c3f50c9 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2306,7 +2306,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2316,7 +2316,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3640,7 +3640,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index bd7ec923e9..74e5099283 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1030,6 +1033,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1120,21 +1127,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1148,21 +1211,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1241,6 +1313,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1688,14 +1848,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2649,6 +2804,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3002,6 +3168,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3637,6 +3865,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 84a0b99311..7f5375fd14 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,48 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +104,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +123,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -402,16 +479,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -436,13 +530,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -465,7 +595,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -545,6 +675,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index cd56714968..2b62e6f47c 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 3a89f8fe1e..aa4c7b3ec5 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -634,6 +636,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 51470dd73e..52f22f2379 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 4f42f342b0..129862c1ca 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1469,7 +1474,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1551,6 +1556,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index f8e2c6e88e..a929cfd8e9 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/pg_am.h"
@@ -5739,6 +5740,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index cd3d6ed60a..e0534c3527 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -290,6 +291,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -391,6 +394,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -721,6 +725,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7007,7 +7015,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7043,7 +7053,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7068,7 +7133,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7107,7 +7174,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7142,7 +7211,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7173,7 +7244,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7207,7 +7280,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7247,6 +7322,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7272,6 +7349,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16544,7 +16623,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16611,6 +16691,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16639,6 +16723,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18620,6 +18719,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 3e11166615..f7f33c35c3 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 1b90cbd9b5..74c457f1e5 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1669,6 +1695,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1683,7 +1710,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2351,6 +2378,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2544,6 +2572,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2611,6 +2640,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2682,6 +2712,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3149,6 +3180,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3164,6 +3196,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3296,16 +3329,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3329,6 +3399,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3379,16 +3453,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3436,6 +3523,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3489,79 +3582,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 62e86a19b6..389e28b13e 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -156,7 +163,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -176,17 +184,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -201,11 +221,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index a2890c1314..9753b0cde2 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+void index_update_collation_versions(Oid relid);
+
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
char relpersistence, int options);
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index a649e44d08..d29ae8e3e6 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10254,6 +10254,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 74106b3731..b12063306e 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1d5450758e..39f84a3844 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -764,10 +764,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index ae95bb38a6..94b4daf4d6 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--u65IjBhB3TIa72Vp
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v19-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v20 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 2 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 256 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1665 insertions(+), 157 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 5cf3c8a836..4d65dbf6dd 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -23005,7 +23005,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system.
+ operating system. An empty string is returned if the version is unknown.
</para>
<para>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 997ef1e12a..4b26838ad0 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -212,6 +212,24 @@
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index c54a7c420d..7209a606fa 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 281f5b7c28..ddcfb7a856 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -78,6 +78,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -138,6 +139,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -438,6 +442,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1591,6 +1670,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1605,8 +1688,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1633,12 +1716,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1694,8 +1783,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1716,8 +1805,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1739,8 +1828,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1773,6 +1867,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1798,10 +1935,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1891,7 +2030,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1979,7 +2119,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2010,7 +2151,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2023,7 +2165,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2036,7 +2179,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2125,7 +2269,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2270,7 +2415,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2292,7 +2439,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2688,8 +2837,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 632c058b80..0e6c3f50c9 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2306,7 +2306,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2316,7 +2316,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3640,7 +3640,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 7cfbdd57db..c8d643a3ed 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1119,21 +1126,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1147,21 +1210,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1240,6 +1312,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1687,14 +1847,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2648,6 +2803,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3001,6 +3167,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3629,6 +3857,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 26a7f1d216..e118e041c1 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,48 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +104,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +123,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +528,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +579,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +644,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +724,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index cd56714968..2b62e6f47c 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 5a110edb07..57e0b6e900 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 4f42f342b0..129862c1ca 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1469,7 +1474,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1551,6 +1556,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 9f1f11d0c1..e40e794b11 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5925,6 +5926,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 4854b9d4d5..02de380900 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -290,6 +291,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -391,6 +394,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -721,6 +725,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7023,7 +7031,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7059,7 +7069,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7084,7 +7149,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7123,7 +7190,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7158,7 +7227,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7189,7 +7260,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7223,7 +7296,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7263,6 +7338,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7288,6 +7365,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16565,7 +16644,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16632,6 +16712,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16660,6 +16744,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18641,6 +18740,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 61c909e06d..62a8e12c1b 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 1b90cbd9b5..74c457f1e5 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1669,6 +1695,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1683,7 +1710,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2351,6 +2378,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2544,6 +2572,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2611,6 +2640,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2682,6 +2712,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3149,6 +3180,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3164,6 +3196,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3296,16 +3329,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3329,6 +3399,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3379,16 +3453,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3436,6 +3523,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3489,79 +3582,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 0ad0597050..42a4583be1 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -156,7 +163,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -176,17 +184,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -205,11 +225,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 4bce3ad8de..9e492217aa 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10333,6 +10333,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1d5450758e..39f84a3844 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -764,10 +764,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index ae95bb38a6..94b4daf4d6 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--Q68bSM7Ycu6FN28Q
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v20-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH 4/6] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. That version is checked against current
version whenever we call get_relation_info for an index or open the parent
table during non-full VACUUM or ANALYZE. Warn that the index may be corrupted
if the versions don't match.
A new flag is added in RelationData to specify that the check has already beed
done to avoid checking and reporting the message multiple time in a backend
lifetime.
Author: Thomas Munro, Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 2 +-
src/backend/catalog/dependency.c | 189 +++++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 191 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 136 +++++++++++--
src/backend/catalog/pg_type.c | 69 +++++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 31 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 ++++-
src/backend/utils/cache/relcache.c | 2 +
src/include/catalog/dependency.h | 22 +-
src/include/catalog/index.h | 2 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++++
.../regress/expected/collate.icu.utf8.out | 130 ++++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 83 ++++++++
24 files changed, 968 insertions(+), 83 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index b2d991ac7f..8414f2bfd9 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21115,7 +21115,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system.
+ operating system. An empty string is returned if the version is unknown.
</para>
<para>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 78c31baa34..93f57cd633 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -77,6 +77,7 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -137,6 +138,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -437,6 +441,80 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version, *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1590,6 +1668,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,9 +1684,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1631,12 +1714,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1690,9 +1779,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, NULL,
+ self_addrs->refs,
self_addrs->numrefs,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,9 +1801,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1735,8 +1826,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1769,6 +1865,46 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /* Record collations from the type itself, or underlying in case of
+ * complex type. Note that if the direct parent is a CollateExpr
+ * node, there's no need to record the type underlying collation if
+ * any. A dependency already exists for the owning relation, and a
+ * change in the collation sort order wouldn't cause any harm as
+ * the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,11 +1929,13 @@ find_expr_references_walker(Node *node,
/*
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
- * a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * a simple constant. However we can save work in the most common case
+ * where the collation is "default", since we know that's pinned, if
+ * the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1887,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1975,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2006,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2019,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2032,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2121,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2266,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2288,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2684,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, NULL, referenced->numrefs,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index e31478bf91..46e9d74a97 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2304,7 +2304,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2314,7 +2314,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3638,7 +3638,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */,
+ false /* don't track versions */);
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1681f61727..9f9281a7af 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -52,6 +52,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -74,6 +75,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -117,6 +119,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1025,6 +1028,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1115,21 +1122,75 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or not,
+ * removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1143,21 +1204,29 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1229,6 +1298,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+static void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2634,6 +2791,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3611,6 +3779,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 3d2b1cc911..0eaefbd032 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 7fdbdf0ae8..7e087fefc4 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,16 +19,21 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -44,34 +49,47 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
- * As recordDependencyOn(), but also capture a version string so that changes
- * in the referenced object can be detected. The meaning of the version
- * string depends on the referenced object. Currently it is used for
- * detecting changes in collation versions.
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
*/
-void
-recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior)
+void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
{
- recordMultipleDependencies(depender, referenced, version, 1, behavior);
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -79,6 +97,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -97,12 +116,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries and
+ * calling CommandCounterIncrement() if the dependencies are
+ * registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -559,6 +615,54 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenceds addresses.
+ */
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 8d7572da51..6e91518694 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -511,6 +512,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 493aa21a14..05709fab77 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -272,28 +272,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d625d17bf4..2b7de111cd 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -634,6 +636,35 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index d82fc5ab8b..f2fc427fc8 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 60dab33fcb..18a2d72f91 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1473,7 +1478,7 @@ pg_newlocale_from_collation(Oid collid)
* NULL (if it doesn't support versions). It must not return NULL for some
* collcollate and not NULL for others.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1511,6 +1516,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index ff70326474..bdf50ffe89 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/pg_am.h"
@@ -5623,6 +5624,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 77cf0612ed..8750bfc36f 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -156,7 +156,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -176,22 +177,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
-extern void recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index a2890c1314..c619d02465 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index e1a5ab3df3..2bf6f868a5 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -335,6 +335,8 @@ extern void GenerateTypeDependencies(Oid typeObjectId,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 44ed04dd3f..3656ea94e8 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..09512c0f66 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,136 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index ae95bb38a6..94b4daf4d6 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..c8f1a620d2 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,89 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--mP3DRpeJDSE+ciuQ
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename*0=0005-Preserve-index-dependencies-on-collation-during-pg_u-v12;
filename*1=".patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v24 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 256 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1666 insertions(+), 157 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 3c83fe6bab..d28512a638 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2301,7 +2301,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2311,7 +2311,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3635,7 +3635,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index cdc01c49c9..210b25d147 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1119,21 +1126,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1147,21 +1210,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1240,6 +1312,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1691,14 +1851,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2652,6 +2807,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3005,6 +3171,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3633,6 +3861,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..b10fc9a241 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,48 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +104,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +123,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +528,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +579,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +644,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +724,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--wac7ysb48OaltWcw
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v24-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v28 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 218 +++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 257 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 206 +++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 31 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1648 insertions(+), 146 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..d6a3cae303 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1566,6 +1645,36 @@ ReleaseDeletionLock(const ObjectAddress *object)
AccessExclusiveLock);
}
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ObjectAddresses *addrs;
+ ListCell *lc;
+
+ if (list_length(collations) == 0)
+ return;
+
+ addrs = new_object_addresses();
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ add_exact_object_address(&referenced, addrs);
+ }
+
+ recordMultipleDependencies(myself, addrs->refs, addrs->numrefs,
+ DEPENDENCY_NORMAL, record_version);
+}
+
/*
* recordDependencyOnExpr - find expression dependencies
*
@@ -1588,6 +1697,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1715,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1743,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1810,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1832,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1855,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1894,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1962,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2057,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2178,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2192,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2206,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2296,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2442,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2466,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2864,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..e5155785b6 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,32 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /*
+ * recordDependencyOnSingleRelExpr gets rid of duplicated entries
+ */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1284,108 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+/* index_check_collation_version
+ * Raise a warning if the recorded and current collation version don't
+ * match.
+*/
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+/* index_check_collation_versions
+ * Check the collation version for all dependencies on the given object.
+ */
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+/* index_update_collation_version
+ * Return the current collation version for the given object.
+ */
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+/* index_update_collation_versions
+ * Record the current collation versions of all dependencies on the given
+ * object.
+ */
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2977,6 +3151,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3841,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..8137e50036 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,14 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +51,24 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +76,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +95,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant dependencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,7 +500,8 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
@@ -459,8 +509,7 @@ changeDependencyFor(Oid classId, Oid objectId,
* Returns the number of records updated.
*/
long
-changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+changeDependenciesOf(Oid classId, Oid oldObjectId, Oid newObjectId)
{
long count = 0;
Relation depRel;
@@ -485,13 +534,46 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
- depform->objid = newObjectId;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -594,6 +676,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..2966339498 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,10 +229,9 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
+long changeDependenciesOf(Oid classId, Oid oldObjectId,
Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--gclmslubawp3yaq7
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v28-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v28 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 218 +++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 257 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 206 +++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 31 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1648 insertions(+), 146 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..d6a3cae303 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1566,6 +1645,36 @@ ReleaseDeletionLock(const ObjectAddress *object)
AccessExclusiveLock);
}
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ObjectAddresses *addrs;
+ ListCell *lc;
+
+ if (list_length(collations) == 0)
+ return;
+
+ addrs = new_object_addresses();
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ add_exact_object_address(&referenced, addrs);
+ }
+
+ recordMultipleDependencies(myself, addrs->refs, addrs->numrefs,
+ DEPENDENCY_NORMAL, record_version);
+}
+
/*
* recordDependencyOnExpr - find expression dependencies
*
@@ -1588,6 +1697,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1715,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1743,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1810,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1832,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1855,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1894,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1962,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2057,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2178,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2192,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2206,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2296,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2442,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2466,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2864,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..e5155785b6 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,32 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /*
+ * recordDependencyOnSingleRelExpr gets rid of duplicated entries
+ */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1284,108 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+/* index_check_collation_version
+ * Raise a warning if the recorded and current collation version don't
+ * match.
+*/
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+/* index_check_collation_versions
+ * Check the collation version for all dependencies on the given object.
+ */
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+/* index_update_collation_version
+ * Return the current collation version for the given object.
+ */
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+/* index_update_collation_versions
+ * Record the current collation versions of all dependencies on the given
+ * object.
+ */
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2977,6 +3151,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3841,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..8137e50036 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,14 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +51,24 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +76,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +95,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant dependencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,7 +500,8 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
@@ -459,8 +509,7 @@ changeDependencyFor(Oid classId, Oid objectId,
* Returns the number of records updated.
*/
long
-changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+changeDependenciesOf(Oid classId, Oid oldObjectId, Oid newObjectId)
{
long count = 0;
Relation depRel;
@@ -485,13 +534,46 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
- depform->objid = newObjectId;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -594,6 +676,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..2966339498 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,10 +229,9 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
+long changeDependenciesOf(Oid classId, Oid oldObjectId,
Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--gclmslubawp3yaq7
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v28-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v27 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 250 +++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 +++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1650 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..5c3a84629f 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1588,6 +1667,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1685,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1713,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1780,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1802,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1825,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1864,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1932,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2027,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2116,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2148,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2162,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2176,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2266,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2412,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2436,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2834,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..5cd3af3053 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1282,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1663,14 +1821,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2977,6 +3130,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3820,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..c1e37f387c 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--o56keubw4jpehgac
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v27-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v25 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1665 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index d279842d3c..93a7584a0c 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2295,7 +2295,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2305,7 +2305,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3629,7 +3629,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index fc088d3f52..e85aeb5502 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1107,19 +1114,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1130,21 +1195,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1223,6 +1297,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1674,14 +1836,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2635,6 +2792,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -2988,6 +3156,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3616,6 +3846,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index fdc63e7dea..5b3c4ab637 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..1b6c78791e 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--lrZ03NoBR/3+SXJZ
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v25-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v28 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 218 +++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 257 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 206 +++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 31 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1648 insertions(+), 146 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..d6a3cae303 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1566,6 +1645,36 @@ ReleaseDeletionLock(const ObjectAddress *object)
AccessExclusiveLock);
}
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ObjectAddresses *addrs;
+ ListCell *lc;
+
+ if (list_length(collations) == 0)
+ return;
+
+ addrs = new_object_addresses();
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ add_exact_object_address(&referenced, addrs);
+ }
+
+ recordMultipleDependencies(myself, addrs->refs, addrs->numrefs,
+ DEPENDENCY_NORMAL, record_version);
+}
+
/*
* recordDependencyOnExpr - find expression dependencies
*
@@ -1588,6 +1697,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1715,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1743,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1810,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1832,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1855,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1894,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1962,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2057,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2178,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2192,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2206,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2296,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2442,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2466,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2864,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..e5155785b6 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,32 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /*
+ * recordDependencyOnSingleRelExpr gets rid of duplicated entries
+ */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1284,108 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+/* index_check_collation_version
+ * Raise a warning if the recorded and current collation version don't
+ * match.
+*/
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+/* index_check_collation_versions
+ * Check the collation version for all dependencies on the given object.
+ */
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+/* index_update_collation_version
+ * Return the current collation version for the given object.
+ */
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+/* index_update_collation_versions
+ * Record the current collation versions of all dependencies on the given
+ * object.
+ */
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2977,6 +3151,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3841,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..8137e50036 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,14 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +51,24 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +76,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +95,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant dependencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,7 +500,8 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
@@ -459,8 +509,7 @@ changeDependencyFor(Oid classId, Oid objectId,
* Returns the number of records updated.
*/
long
-changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+changeDependenciesOf(Oid classId, Oid oldObjectId, Oid newObjectId)
{
long count = 0;
Relation depRel;
@@ -485,13 +534,46 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
- depform->objid = newObjectId;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -594,6 +676,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..2966339498 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,10 +229,9 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
+long changeDependenciesOf(Oid classId, Oid oldObjectId,
Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--gclmslubawp3yaq7
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v28-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v25 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1665 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index d279842d3c..93a7584a0c 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2295,7 +2295,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2305,7 +2305,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3629,7 +3629,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index fc088d3f52..e85aeb5502 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1107,19 +1114,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1130,21 +1195,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1223,6 +1297,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1674,14 +1836,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2635,6 +2792,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -2988,6 +3156,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3616,6 +3846,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index fdc63e7dea..5b3c4ab637 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..1b6c78791e 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--lrZ03NoBR/3+SXJZ
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v25-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v27 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 250 +++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 +++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1650 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..5c3a84629f 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1588,6 +1667,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1685,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1713,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1780,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1802,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1825,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1864,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1932,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2027,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2116,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2148,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2162,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2176,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2266,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2412,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2436,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2834,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..5cd3af3053 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1282,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1663,14 +1821,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2977,6 +3130,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3820,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..c1e37f387c 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--o56keubw4jpehgac
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v27-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v25 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1665 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index d279842d3c..93a7584a0c 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2295,7 +2295,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2305,7 +2305,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3629,7 +3629,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index fc088d3f52..e85aeb5502 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1107,19 +1114,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1130,21 +1195,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1223,6 +1297,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1674,14 +1836,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2635,6 +2792,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -2988,6 +3156,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3616,6 +3846,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index fdc63e7dea..5b3c4ab637 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..1b6c78791e 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--lrZ03NoBR/3+SXJZ
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v25-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v28 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 218 +++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 257 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 206 +++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 31 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1648 insertions(+), 146 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..d6a3cae303 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1566,6 +1645,36 @@ ReleaseDeletionLock(const ObjectAddress *object)
AccessExclusiveLock);
}
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ObjectAddresses *addrs;
+ ListCell *lc;
+
+ if (list_length(collations) == 0)
+ return;
+
+ addrs = new_object_addresses();
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ add_exact_object_address(&referenced, addrs);
+ }
+
+ recordMultipleDependencies(myself, addrs->refs, addrs->numrefs,
+ DEPENDENCY_NORMAL, record_version);
+}
+
/*
* recordDependencyOnExpr - find expression dependencies
*
@@ -1588,6 +1697,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1715,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1743,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1810,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1832,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1855,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1894,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1962,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2057,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2178,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2192,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2206,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2296,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2442,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2466,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2864,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..e5155785b6 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,32 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /*
+ * recordDependencyOnSingleRelExpr gets rid of duplicated entries
+ */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1284,108 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+/* index_check_collation_version
+ * Raise a warning if the recorded and current collation version don't
+ * match.
+*/
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+/* index_check_collation_versions
+ * Check the collation version for all dependencies on the given object.
+ */
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+/* index_update_collation_version
+ * Return the current collation version for the given object.
+ */
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+/* index_update_collation_versions
+ * Record the current collation versions of all dependencies on the given
+ * object.
+ */
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2977,6 +3151,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3841,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..8137e50036 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,14 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +51,24 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +76,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +95,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant dependencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,7 +500,8 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
@@ -459,8 +509,7 @@ changeDependencyFor(Oid classId, Oid objectId,
* Returns the number of records updated.
*/
long
-changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+changeDependenciesOf(Oid classId, Oid oldObjectId, Oid newObjectId)
{
long count = 0;
Relation depRel;
@@ -485,13 +534,46 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
- depform->objid = newObjectId;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -594,6 +676,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..2966339498 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,10 +229,9 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
+long changeDependenciesOf(Oid classId, Oid oldObjectId,
Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--gclmslubawp3yaq7
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v28-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v24 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 256 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1666 insertions(+), 157 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 3c83fe6bab..d28512a638 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2301,7 +2301,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2311,7 +2311,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3635,7 +3635,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index cdc01c49c9..210b25d147 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1119,21 +1126,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1147,21 +1210,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1240,6 +1312,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1691,14 +1851,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2652,6 +2807,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3005,6 +3171,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3633,6 +3861,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..b10fc9a241 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,48 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +104,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +123,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +528,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +579,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +644,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +724,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--wac7ysb48OaltWcw
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v24-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v28 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 218 +++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 257 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 206 +++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 31 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1648 insertions(+), 146 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..d6a3cae303 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1566,6 +1645,36 @@ ReleaseDeletionLock(const ObjectAddress *object)
AccessExclusiveLock);
}
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ObjectAddresses *addrs;
+ ListCell *lc;
+
+ if (list_length(collations) == 0)
+ return;
+
+ addrs = new_object_addresses();
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ add_exact_object_address(&referenced, addrs);
+ }
+
+ recordMultipleDependencies(myself, addrs->refs, addrs->numrefs,
+ DEPENDENCY_NORMAL, record_version);
+}
+
/*
* recordDependencyOnExpr - find expression dependencies
*
@@ -1588,6 +1697,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1715,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1743,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1810,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1832,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1855,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1894,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1962,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2057,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2178,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2192,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2206,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2296,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2442,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2466,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2864,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..e5155785b6 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,32 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /*
+ * recordDependencyOnSingleRelExpr gets rid of duplicated entries
+ */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1284,108 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+/* index_check_collation_version
+ * Raise a warning if the recorded and current collation version don't
+ * match.
+*/
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+/* index_check_collation_versions
+ * Check the collation version for all dependencies on the given object.
+ */
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+/* index_update_collation_version
+ * Return the current collation version for the given object.
+ */
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+/* index_update_collation_versions
+ * Record the current collation versions of all dependencies on the given
+ * object.
+ */
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2977,6 +3151,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3841,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..8137e50036 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,14 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +51,24 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +76,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +95,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant dependencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,7 +500,8 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
@@ -459,8 +509,7 @@ changeDependencyFor(Oid classId, Oid objectId,
* Returns the number of records updated.
*/
long
-changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+changeDependenciesOf(Oid classId, Oid oldObjectId, Oid newObjectId)
{
long count = 0;
Relation depRel;
@@ -485,13 +534,46 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
- depform->objid = newObjectId;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -594,6 +676,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..2966339498 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,10 +229,9 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
+long changeDependenciesOf(Oid classId, Oid oldObjectId,
Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--gclmslubawp3yaq7
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v28-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v28 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 218 +++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 257 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 206 +++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 31 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1648 insertions(+), 146 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..d6a3cae303 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1566,6 +1645,36 @@ ReleaseDeletionLock(const ObjectAddress *object)
AccessExclusiveLock);
}
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ObjectAddresses *addrs;
+ ListCell *lc;
+
+ if (list_length(collations) == 0)
+ return;
+
+ addrs = new_object_addresses();
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ add_exact_object_address(&referenced, addrs);
+ }
+
+ recordMultipleDependencies(myself, addrs->refs, addrs->numrefs,
+ DEPENDENCY_NORMAL, record_version);
+}
+
/*
* recordDependencyOnExpr - find expression dependencies
*
@@ -1588,6 +1697,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1715,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1743,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1810,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1832,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1855,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1894,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1962,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2057,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2178,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2192,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2206,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2296,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2442,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2466,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2864,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..e5155785b6 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,32 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /*
+ * recordDependencyOnSingleRelExpr gets rid of duplicated entries
+ */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1284,108 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+/* index_check_collation_version
+ * Raise a warning if the recorded and current collation version don't
+ * match.
+*/
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+/* index_check_collation_versions
+ * Check the collation version for all dependencies on the given object.
+ */
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+/* index_update_collation_version
+ * Return the current collation version for the given object.
+ */
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+/* index_update_collation_versions
+ * Record the current collation versions of all dependencies on the given
+ * object.
+ */
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2977,6 +3151,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3841,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..8137e50036 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,14 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +51,24 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +76,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +95,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant dependencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,7 +500,8 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
@@ -459,8 +509,7 @@ changeDependencyFor(Oid classId, Oid objectId,
* Returns the number of records updated.
*/
long
-changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+changeDependenciesOf(Oid classId, Oid oldObjectId, Oid newObjectId)
{
long count = 0;
Relation depRel;
@@ -485,13 +534,46 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
- depform->objid = newObjectId;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -594,6 +676,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..2966339498 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,10 +229,9 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
+long changeDependenciesOf(Oid classId, Oid oldObjectId,
Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--gclmslubawp3yaq7
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v28-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v24 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 256 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1666 insertions(+), 157 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 3c83fe6bab..d28512a638 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2301,7 +2301,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2311,7 +2311,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3635,7 +3635,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index cdc01c49c9..210b25d147 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1119,21 +1126,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1147,21 +1210,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1240,6 +1312,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1691,14 +1851,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2652,6 +2807,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3005,6 +3171,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3633,6 +3861,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..b10fc9a241 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,48 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +104,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +123,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +528,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +579,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +644,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +724,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--wac7ysb48OaltWcw
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v24-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v14 4/7] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 2 +-
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 189 +++++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 191 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 136 +++++++++++--
src/backend/catalog/pg_type.c | 69 +++++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 31 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 ++++-
src/backend/utils/cache/relcache.c | 2 +
src/include/catalog/dependency.h | 22 +-
src/include/catalog/index.h | 2 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++++
.../regress/expected/collate.icu.utf8.out | 130 ++++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 83 ++++++++
25 files changed, 974 insertions(+), 83 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index b2d991ac7f..8414f2bfd9 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21115,7 +21115,7 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
<para>
<function>pg_collation_actual_version</function> returns the actual
version of the collation object as it is currently installed in the
- operating system.
+ operating system. An empty string is returned if the version is unknown.
</para>
<para>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index c54a7c420d..7209a606fa 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 660033b9c1..c0e36ef4cd 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -78,6 +78,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -138,6 +139,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -438,6 +442,80 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version, *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1591,6 +1669,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1603,9 +1685,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1632,12 +1715,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,9 +1780,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, NULL,
+ self_addrs->refs,
self_addrs->numrefs,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1712,9 +1802,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1827,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1866,46 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /* Record collations from the type itself, or underlying in case of
+ * complex type. Note that if the direct parent is a CollateExpr
+ * node, there's no need to record the type underlying collation if
+ * any. A dependency already exists for the owning relation, and a
+ * change in the collation sort order wouldn't cause any harm as
+ * the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1794,11 +1930,13 @@ find_expr_references_walker(Node *node,
/*
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
- * a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * a simple constant. However we can save work in the most common case
+ * where the collation is "default", since we know that's pinned, if
+ * the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2026,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2115,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2147,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2161,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2175,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2265,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2411,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2435,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2833,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, NULL, referenced->numrefs,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9d9e915979..4546789446 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2304,7 +2304,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2314,7 +2314,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3638,7 +3638,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */,
+ false /* don't track versions */);
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 76fd938ce3..369b5d43f1 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -52,6 +52,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -74,6 +75,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -117,6 +119,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1025,6 +1028,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1115,21 +1122,75 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or not,
+ * removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1143,21 +1204,29 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1229,6 +1298,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+static void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2635,6 +2792,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3623,6 +3791,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 7fdbdf0ae8..7e087fefc4 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,16 +19,21 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -44,34 +49,47 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
- * As recordDependencyOn(), but also capture a version string so that changes
- * in the referenced object can be detected. The meaning of the version
- * string depends on the referenced object. Currently it is used for
- * detecting changes in collation versions.
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
*/
-void
-recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior)
+void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
{
- recordMultipleDependencies(depender, referenced, version, 1, behavior);
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -79,6 +97,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -97,12 +116,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries and
+ * calling CommandCounterIncrement() if the dependencies are
+ * registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -559,6 +615,54 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenceds addresses.
+ */
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index cd56714968..fe14b6bd38 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d625d17bf4..2b7de111cd 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -634,6 +636,35 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index d82fc5ab8b..f2fc427fc8 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 60dab33fcb..18a2d72f91 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1473,7 +1478,7 @@ pg_newlocale_from_collation(Oid collid)
* NULL (if it doesn't support versions). It must not return NULL for some
* collcollate and not NULL for others.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1511,6 +1516,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index ff70326474..bdf50ffe89 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/pg_am.h"
@@ -5623,6 +5624,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 77cf0612ed..8750bfc36f 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -156,7 +156,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -176,22 +177,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
-extern void recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index a2890c1314..c619d02465 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 97890946c5..3eb5ec4fcd 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -350,6 +350,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 44ed04dd3f..3656ea94e8 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..09512c0f66 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,136 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index ae95bb38a6..94b4daf4d6 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..c8f1a620d2 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,89 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--wRRV7LY7NUeQGEoC
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v14-0005-Preserve-index-dependencies-on-collation-during-.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v23 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 256 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1666 insertions(+), 157 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index dd1c1e70f7..8413db4adc 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25385,7 +25385,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 319d613296..bfa6f012f3 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index e393c93a45..79528bc589 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2306,7 +2306,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2316,7 +2316,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3640,7 +3640,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index cdc01c49c9..210b25d147 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1119,21 +1126,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1147,21 +1210,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1240,6 +1312,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1691,14 +1851,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2652,6 +2807,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3005,6 +3171,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3633,6 +3861,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..b10fc9a241 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,48 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +104,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +123,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +528,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +579,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +644,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +724,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index cd56714968..2b62e6f47c 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index aec1f435b0..6897a1f80e 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16559,7 +16638,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16626,6 +16706,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16654,6 +16738,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18635,6 +18734,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 61f2c2f5b4..12363134aa 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10331,6 +10331,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--MGYHOYXEY6WxJCY8
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v23-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v25 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1665 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index d279842d3c..93a7584a0c 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2295,7 +2295,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2305,7 +2305,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3629,7 +3629,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index fc088d3f52..e85aeb5502 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1107,19 +1114,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1130,21 +1195,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1223,6 +1297,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1674,14 +1836,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2635,6 +2792,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -2988,6 +3156,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3616,6 +3846,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index fdc63e7dea..5b3c4ab637 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..1b6c78791e 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--lrZ03NoBR/3+SXJZ
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v25-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v24 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 256 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1666 insertions(+), 157 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 3c83fe6bab..d28512a638 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2301,7 +2301,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2311,7 +2311,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3635,7 +3635,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index cdc01c49c9..210b25d147 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1119,21 +1126,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1147,21 +1210,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1240,6 +1312,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1691,14 +1851,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2652,6 +2807,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3005,6 +3171,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3633,6 +3861,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..b10fc9a241 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,48 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +104,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +123,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +528,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +579,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +644,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +724,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--wac7ysb48OaltWcw
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v24-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v27 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 250 +++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 +++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1650 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..5c3a84629f 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1588,6 +1667,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1685,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1713,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1780,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1802,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1825,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1864,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1932,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2027,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2116,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2148,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2162,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2176,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2266,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2412,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2436,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2834,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..5cd3af3053 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1282,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1663,14 +1821,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2977,6 +3130,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3820,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..c1e37f387c 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--o56keubw4jpehgac
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v27-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v24 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 256 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1666 insertions(+), 157 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 3c83fe6bab..d28512a638 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2301,7 +2301,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2311,7 +2311,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3635,7 +3635,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index cdc01c49c9..210b25d147 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1119,21 +1126,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1147,21 +1210,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1240,6 +1312,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1691,14 +1851,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2652,6 +2807,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3005,6 +3171,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3633,6 +3861,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..b10fc9a241 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,48 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +104,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +123,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +528,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +579,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +644,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +724,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--wac7ysb48OaltWcw
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v24-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v27 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 250 +++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 +++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1650 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..5c3a84629f 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1588,6 +1667,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1685,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1713,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1780,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1802,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1825,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1864,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1932,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2027,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2116,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2148,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2162,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2176,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2266,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2412,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2436,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2834,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..5cd3af3053 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1282,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1663,14 +1821,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2977,6 +3130,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3820,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..c1e37f387c 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--o56keubw4jpehgac
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v27-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v24 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 256 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1666 insertions(+), 157 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 3c83fe6bab..d28512a638 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2301,7 +2301,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2311,7 +2311,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3635,7 +3635,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index cdc01c49c9..210b25d147 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1119,21 +1126,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1147,21 +1210,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1240,6 +1312,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1691,14 +1851,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2652,6 +2807,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3005,6 +3171,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3633,6 +3861,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..b10fc9a241 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,48 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +104,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +123,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +528,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +579,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +644,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +724,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--wac7ysb48OaltWcw
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v24-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v24 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 256 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1666 insertions(+), 157 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 3c83fe6bab..d28512a638 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2301,7 +2301,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2311,7 +2311,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3635,7 +3635,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index cdc01c49c9..210b25d147 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1119,21 +1126,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1147,21 +1210,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1240,6 +1312,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1691,14 +1851,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2652,6 +2807,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3005,6 +3171,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3633,6 +3861,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..b10fc9a241 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,48 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +104,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +123,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +528,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +579,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +644,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +724,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--wac7ysb48OaltWcw
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v24-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v25 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 254 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1665 insertions(+), 156 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index d279842d3c..93a7584a0c 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2295,7 +2295,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2305,7 +2305,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3629,7 +3629,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index fc088d3f52..e85aeb5502 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1107,19 +1114,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1130,21 +1195,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1223,6 +1297,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1674,14 +1836,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2635,6 +2792,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -2988,6 +3156,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3616,6 +3846,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index fdc63e7dea..5b3c4ab637 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..1b6c78791e 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,46 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +102,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +121,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +526,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +577,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +642,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +722,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--lrZ03NoBR/3+SXJZ
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v25-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v24 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when creating
or rebuilding an index. The version is checked against the current version
whenever we call get_relation_info for an index or open the parent table
during non-full VACUUM or ANALYZE. Warn that the index may be corrupted if
the versions don't match.
A new flag is added in RelationData to specify that the check has already been
done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro and Julien Rouhaud
Reviewed-by: Peter Eisentraut, Laurenz Albe
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 188 +++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 265 ++++++++++++++++--
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 256 ++++++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 +++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 ++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 32 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 +++++++
37 files changed, 1666 insertions(+), 157 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 69fe242761..0bb1d6c15f 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25419,7 +25419,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..9be2547d16 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ older than 13, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to mark all indexes as using the currently installed version.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 070b27c692..19d6255ca7 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1586,6 +1665,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1600,8 +1683,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1628,12 +1711,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1689,8 +1778,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,8 +1800,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1734,8 +1823,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1768,6 +1862,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,10 +1930,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1886,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1974,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2005,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2018,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2031,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2120,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2265,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2287,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2683,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 3c83fe6bab..d28512a638 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2301,7 +2301,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2311,7 +2311,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3635,7 +3635,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index cdc01c49c9..210b25d147 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -119,6 +121,7 @@ static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
bool immediate,
bool isvalid,
bool isready);
+static bool index_depends_stable_coll_order(Oid amoid);
static void index_update_stats(Relation rel,
bool hasindex,
double reltuples);
@@ -1029,6 +1032,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
myself.classId = RelationRelationId;
myself.objectId = indexRelationId;
@@ -1119,21 +1126,77 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * For deterministic transaction, only track the version if the AM
+ * relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ bool track_version;
+
+ track_version = index_depends_stable_coll_order(indexInfo->ii_Am);
+
+ recordDependencyOnCollations(&myself, determ_colls, track_version);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1147,21 +1210,30 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1240,6 +1312,94 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -1691,14 +1851,9 @@ index_concurrently_swap(Oid newIndexId, Oid oldIndexId, const char *oldName)
/*
* Swap all dependencies of and on the old index to the new one, and
- * vice-versa. Note that a call to CommandCounterIncrement() would cause
- * duplicate entries in pg_depend, so this should not be done.
+ * vice-versa.
*/
- changeDependenciesOf(RelationRelationId, newIndexId, oldIndexId);
- changeDependenciesOn(RelationRelationId, newIndexId, oldIndexId);
-
- changeDependenciesOf(RelationRelationId, oldIndexId, newIndexId);
- changeDependenciesOn(RelationRelationId, oldIndexId, newIndexId);
+ swapDependencies(RelationRelationId, newIndexId, oldIndexId);
/*
* Copy over statistics from old to new index
@@ -2652,6 +2807,17 @@ FormIndexDatum(IndexInfo *indexInfo,
elog(ERROR, "wrong number of index expressions");
}
+/*
+ * Returns whether the given index access method depend on a stable collation
+ * order.
+ */
+static bool
+index_depends_stable_coll_order(Oid amoid)
+{
+ return (amoid != HASH_AM_OID &&
+ strcmp(get_am_name(amoid), "bloom") != 0);
+}
+
/*
* index_update_stats --- update pg_class entry after CREATE INDEX or REINDEX
@@ -3005,6 +3171,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3633,6 +3861,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 90932be831..3cc0f6651c 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5d45e706cb..b10fc9a241 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,18 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static long changeDependenciesOf(Oid classId, Oid oldObjectId,
+ Oid newObjectId, bool preserve_version);
+static long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+ Oid newRefObjectId);
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +55,48 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
+}
+
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, record_version);
+ }
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +104,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +123,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,16 +528,33 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Swap all dependencies of and on the old index to the new one, and
+ * vice-versa, while preserving any referenced version for the original owners.
+ * Note that a call to CommandCounterIncrement() would cause duplicate entries
+ * in pg_depend, so this should not be done.
+ */
+void
+swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId)
+{
+ changeDependenciesOf(classId, firstObjectId, secondObjectId, true);
+ changeDependenciesOn(classId, firstObjectId, secondObjectId);
+
+ changeDependenciesOf(classId, secondObjectId, firstObjectId, true);
+ changeDependenciesOn(classId, secondObjectId, firstObjectId);
+}
+
+/*
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+ Oid newObjectId, bool preserve_version)
{
long count = 0;
Relation depRel;
@@ -485,13 +579,49 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- depform->objid = newObjectId;
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ if (preserve_version)
+ {
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ }
+
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -514,7 +644,7 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
*
* Returns the number of records updated.
*/
-long
+static long
changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId)
{
@@ -594,6 +724,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 78eceda848..d4eccf163f 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -273,28 +273,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d32de23e62..65e65abb9a 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 2c5f5bfa37..4e116485c6 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -148,6 +150,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1639,7 +1644,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1721,6 +1726,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 18f2ee8226..07ff39f5aa 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -208,3 +209,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 0b9eb00d2d..62fefc4fcc 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5926,6 +5927,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 8c0cedcd98..dc82b076ef 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2f6bfede8e..f41acebe38 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7014,7 +7022,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7050,7 +7060,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 130000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7075,7 +7140,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7114,7 +7181,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7149,7 +7218,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7180,7 +7251,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7214,7 +7287,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7254,6 +7329,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7279,6 +7356,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16536,7 +16615,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16603,6 +16683,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16631,6 +16715,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18612,6 +18711,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending ono
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0c2fcfb3a9..2faab3b659 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index e116235769..886d123e25 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -916,9 +935,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1180,6 +1200,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1205,6 +1226,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1240,6 +1262,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1262,6 +1285,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1283,6 +1307,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1304,6 +1329,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1670,6 +1696,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1684,7 +1711,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2352,6 +2379,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2545,6 +2573,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2613,6 +2642,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2684,6 +2714,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3151,6 +3182,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3166,6 +3198,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3298,16 +3331,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3331,6 +3401,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3381,16 +3455,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3438,6 +3525,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3491,79 +3584,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..4949044041 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,11 +229,7 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
- Oid newRefObjectId);
+extern void swapDependencies(Oid classId, Oid firstObjectId, Oid secondObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 38295aca48..3a9bb0d783 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10343,6 +10343,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 1407359aef..f39b9433a8 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..db386c1b09 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+---------------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | version not tracked
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--wac7ysb48OaltWcw
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v24-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH v28 3/5] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. The version is checked against the
current version whenever we call get_relation_info for an index or open
the parent table during non-full VACUUM or ANALYZE. Warn that the index
may be corrupted if the versions don't match.
Add a new flag is added to RelationData to record that the check has
already been done to avoid repetition.
Thanks to Doug Doole, Peter Eisentraut, Christoph Berg, Laurenz Albe,
Michael Paquier, Robert Haas, Tom Lane and others for design discussion.
Author: Thomas Munro <[email protected]>
Author: Julien Rouhaud <[email protected]>
Reviewed-by: Peter Eisentraut <[email protected]>
Reviewed-by: Laurenz Albe <[email protected]>
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
doc/src/sgml/func.sgml | 3 +-
doc/src/sgml/ref/pgupgrade.sgml | 18 ++
doc/src/sgml/ref/reindex.sgml | 6 +
src/backend/catalog/dependency.c | 218 +++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 257 +++++++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 206 +++++++++++++-
src/backend/catalog/pg_type.c | 69 +++++
src/backend/commands/collationcmds.c | 22 +-
src/backend/commands/vacuum.c | 32 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 52 +++-
src/backend/utils/adt/pg_upgrade_support.c | 25 ++
src/backend/utils/cache/relcache.c | 2 +
src/bin/pg_dump/Makefile | 2 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 186 ++++++++++++-
src/bin/pg_dump/pg_dump.h | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 248 +++++++++++++----
src/bin/pg_upgrade/dump.c | 4 +-
src/bin/pg_upgrade/option.c | 7 +
src/bin/pg_upgrade/pg_upgrade.h | 2 +
src/include/catalog/dependency.h | 31 ++-
src/include/catalog/index.h | 7 +
src/include/catalog/pg_proc.dat | 4 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 2 +-
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 ++++++
src/test/perl/PostgresNode.pm | 6 +-
.../regress/expected/collate.icu.utf8.out | 157 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 106 ++++++++
37 files changed, 1648 insertions(+), 146 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 8432e790f2..8465925b26 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -25438,7 +25438,8 @@ postgres=# SELECT * FROM pg_walfile_name_offset(pg_stop_backup());
</para>
<para>
Returns the actual version of the collation object as it is currently
- installed in the operating system.
+ installed in the operating system. An empty string is returned if the
+ version is unknown.
</para></entry>
</row>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
index 6779a5bddc..4f789e49fb 100644
--- a/doc/src/sgml/ref/pgupgrade.sgml
+++ b/doc/src/sgml/ref/pgupgrade.sgml
@@ -215,6 +215,24 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--collation-binary-compatible</option></term>
+ <listitem>
+ <para>
+ After upgrading from a release of <productname>PostgreSQL</productname>
+ before 14, warnings may be reported when collation-dependent
+ indexes are first accessed. This is because
+ <application>pg_upgrade</application> records the collation versions
+ as unknown, so <productname>PostgreSQL</productname> considers the
+ indexes to be potentially corrupted. If you're certain that the
+ collation definitions used by your operating system or ICU haven't
+ changed since all indexes were created, you can use this flag
+ to record that the indexes match the currently installed collations.
+ Otherwise, see <xref linkend="sql-reindex"/>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-?</option></term>
<term><option>--help</option></term>
diff --git a/doc/src/sgml/ref/reindex.sgml b/doc/src/sgml/ref/reindex.sgml
index aac5d5be23..48726d261d 100644
--- a/doc/src/sgml/ref/reindex.sgml
+++ b/doc/src/sgml/ref/reindex.sgml
@@ -39,6 +39,12 @@ REINDEX [ ( <replaceable class="parameter">option</replaceable> [, ...] ) ] { IN
<itemizedlist>
<listitem>
+ <para>
+ The index depends on the sort order of a collation, and the definition
+ of the collation has changed. This can cause an index to fail to
+ find a key that is present in the index. On some systems, collation
+ versions are tracked in order to generate warnings when this happens.
+ </para>
<para>
An index has become corrupted, and no longer contains valid
data. Although in theory this should never happen, in
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 1a927377e7..d6a3cae303 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -76,6 +76,7 @@
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
#include "utils/acl.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -136,6 +137,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -434,6 +438,81 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version,
+ *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1566,6 +1645,36 @@ ReleaseDeletionLock(const ObjectAddress *object)
AccessExclusiveLock);
}
+/*
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
+ */
+void
+recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version)
+{
+ ObjectAddresses *addrs;
+ ListCell *lc;
+
+ if (list_length(collations) == 0)
+ return;
+
+ addrs = new_object_addresses();
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ ObjectAddressSet(referenced, CollationRelationId, lfirst_oid(lc));
+
+ add_exact_object_address(&referenced, addrs);
+ }
+
+ recordMultipleDependencies(myself, addrs->refs, addrs->numrefs,
+ DEPENDENCY_NORMAL, record_version);
+}
+
/*
* recordDependencyOnExpr - find expression dependencies
*
@@ -1588,6 +1697,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,8 +1715,8 @@ recordDependencyOnExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1630,12 +1743,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1691,8 +1810,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
self_addrs->refs,
self_addrs->numrefs,
- NULL,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1713,8 +1832,8 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
recordMultipleDependencies(depender,
context.addrs->refs,
context.addrs->numrefs,
- NULL,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1736,8 +1855,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1770,6 +1894,49 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /*
+ * Record collations from the type itself, or underlying in case
+ * of complex type. Note that if the direct parent is a
+ * CollateExpr node, there's no need to record the type underlying
+ * collation if any. A dependency already exists for the owning
+ * relation, and a change in the collation sort order wouldn't
+ * cause any harm as the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations
+ */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1795,10 +1962,12 @@ find_expr_references_walker(Node *node,
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
* a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * case where the collation is "default", since we know that's pinned,
+ * if the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1888,7 +2057,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1976,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2007,7 +2178,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2020,7 +2192,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2033,7 +2206,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2122,7 +2296,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2267,7 +2442,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2289,7 +2466,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2685,8 +2864,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, referenced->numrefs, NULL,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..b44dde7bbd 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2344,7 +2344,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2354,7 +2354,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3678,7 +3678,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */ ,
+ false /* don't track versions */ );
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..e5155785b6 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -53,6 +53,7 @@
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
#include "catalog/storage.h"
+#include "commands/defrem.h"
#include "commands/event_trigger.h"
#include "commands/progress.h"
#include "commands/tablecmds.h"
@@ -75,6 +76,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -1018,6 +1020,10 @@ index_create(Relation heapRelation,
{
ObjectAddress myself,
referenced;
+ ListCell *lc;
+ List *colls = NIL,
+ *determ_colls = NIL,
+ *nondeterm_colls = NIL;
ObjectAddressSet(myself, RelationRelationId, indexRelationId);
@@ -1096,19 +1102,74 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /* First, get all dependencies on collations for all index keys. */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
+ colls = lappend_oid(colls, colloid);
+ else
{
- ObjectAddressSet(referenced, CollationRelationId,
- collationObjectId[i]);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
+
+ colls = list_concat(colls,
+ GetTypeCollations(att->atttypid, false));
}
}
+ /*
+ * Then split the dependencies on whether they're deterministic or
+ * not, removing any duplicates.
+ */
+ foreach(lc, colls)
+ {
+ Oid c = lfirst_oid(lc);
+
+ if (!OidIsValid(c))
+ continue;
+
+ if (get_collation_isdeterministic(c))
+ determ_colls = list_append_unique_oid(determ_colls, c);
+ else
+ nondeterm_colls = list_append_unique_oid(nondeterm_colls, c);
+ }
+
+ /*
+ * XXX For deterministic transaction, se should only track the version
+ * if the AM relies on a stable ordering.
+ */
+ if (determ_colls)
+ {
+ /* XXX check if the AM relies on a stable ordering */
+ recordDependencyOnCollations(&myself, determ_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
+ /*
+ * We always record the version for dependency on non-deterministic
+ * collations.
+ */
+ if (nondeterm_colls)
+ {
+ recordDependencyOnCollations(&myself, nondeterm_colls, true);
+
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1119,21 +1180,32 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /*
+ * recordDependencyOnSingleRelExpr gets rid of duplicated entries
+ */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicated entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1212,6 +1284,108 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+/* index_check_collation_version
+ * Raise a warning if the recorded and current collation version don't
+ * match.
+*/
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || (strcmp(version, "") == 0 && strcmp(current_version,
+ "") != 0))
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+/* index_check_collation_versions
+ * Check the collation version for all dependencies on the given object.
+ */
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+/* index_update_collation_version
+ * Return the current collation version for the given object.
+ */
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+/* index_update_collation_versions
+ * Record the current collation versions of all dependencies on the given
+ * object.
+ */
+void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -2977,6 +3151,68 @@ index_build(Relation heapRelation,
SetUserIdAndSecContext(save_userid, save_sec_context);
}
+static char *
+index_force_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ NewCollationVersionDependency *forced_dependency;
+
+ forced_dependency = (NewCollationVersionDependency *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /*
+ * We only care about dependencies on a specific collation if a valid Oid
+ * was given.
+ */
+ if (OidIsValid(forced_dependency->oid) &&
+ otherObject->objectId != forced_dependency->oid)
+ return NULL;
+
+ return forced_dependency->version;
+}
+
+/* index_force_collation_versions
+ *
+ * Override collation version dependencies for the given index. If no
+ * collation identifier is specified, all dependent collation should be
+ * reset to an unknown version dependency, and no version should be provided
+ * either.
+ */
+void
+index_force_collation_versions(Oid indexid, Oid coll, char *version)
+{
+ Relation index;
+ ObjectAddress object;
+ NewCollationVersionDependency forced_dependency;
+
+ Assert(version);
+
+ index = relation_open(indexid, AccessExclusiveLock);
+
+ if (index->rd_rel->relkind != RELKIND_INDEX)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ (errmsg("\"%s\" is not an index", get_rel_name(indexid)))));
+
+ forced_dependency.oid = InvalidOid;
+ forced_dependency.version = version;
+
+ object.classId = RelationRelationId;
+ object.objectId = indexid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_force_collation_version,
+ &forced_dependency);
+
+ /* Invalidate the index relcache */
+ CacheInvalidateRelcache(index);
+
+ relation_close(index, NoLock);
+}
+
/*
* IndexCheckExclusion - verify that a new exclusion constraint is satisfied
*
@@ -3605,6 +3841,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 6a6b2cb8c0..3a9252f345 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -346,7 +346,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index d86f85d142..8137e50036 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,6 +19,7 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
@@ -27,9 +28,14 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
+static char *getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -45,19 +51,24 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, 1, NULL, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
* Record multiple dependencies (of the same kind) for a single dependent
* object. This has a little less overhead than recording each separately.
+ *
+ * If track_version is true, then a record could be added even if the referenced
+ * dependency is pinned, and the dependency version will be retrieved according
+ * to the referenced object kind.
+ * For now, only collation version is supported.
*/
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -65,6 +76,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -83,12 +95,49 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation is supported. */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant dependencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries
+ * and calling CommandCounterIncrement() if the dependencies
+ * are registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version.
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;
+ version = get_collation_version_for_oid(referenced->objectId);
+ Assert(version);
+ }
+ }
+ else
+ Assert(!version);
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -451,7 +500,8 @@ changeDependencyFor(Oid classId, Oid objectId,
}
/*
- * Adjust all dependency records to come from a different object of the same type
+ * Adjust all dependency records to come from a different object of the same
+ * type, optionally preserving the original referenced version.
*
* classId/oldObjectId specify the old referencing object.
* newObjectId is the new referencing object (must be of class classId).
@@ -459,8 +509,7 @@ changeDependencyFor(Oid classId, Oid objectId,
* Returns the number of records updated.
*/
long
-changeDependenciesOf(Oid classId, Oid oldObjectId,
- Oid newObjectId)
+changeDependenciesOf(Oid classId, Oid oldObjectId, Oid newObjectId)
{
long count = 0;
Relation depRel;
@@ -485,13 +534,46 @@ changeDependenciesOf(Oid classId, Oid oldObjectId,
while (HeapTupleIsValid((tup = systable_getnext(scan))))
{
Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+ bool isnull = true;
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replaces, 0, sizeof(replaces));
- /* make a modifiable copy */
- tup = heap_copytuple(tup);
- depform = (Form_pg_depend) GETSTRUCT(tup);
+ values[Anum_pg_depend_objid - 1] = newObjectId;
+ replaces[Anum_pg_depend_objid - 1] = true;
- depform->objid = newObjectId;
+ /*
+ * We assume that a version would exist for both the old and new
+ * object or none.
+ */
+ heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ if (!isnull)
+ {
+ ObjectAddress depender,
+ referenced;
+ char *version;
+
+ ObjectAddressSubSet(depender, depform->classid,
+ newObjectId, depform->objsubid);
+ ObjectAddressSubSet(referenced, depform->refclassid,
+ depform->refobjid, depform->refobjsubid);
+
+ version = getDependencyVersion(&depender, &referenced);
+ if (version)
+ values[Anum_pg_depend_refobjversion - 1] = CStringGetTextDatum(version);
+ else
+ nulls[Anum_pg_depend_refobjversion - 1] = true;
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+ }
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
CatalogTupleUpdate(depRel, &tup->t_self, tup);
heap_freetuple(tup);
@@ -594,6 +676,104 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenced addresses.
+ */
+static bool
+dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
+static char *
+getDependencyVersion(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ Datum depversion;
+ char *version = NULL;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ bool isnull;
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+ version = isnull ? NULL : TextDatumGetCString(depversion);
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return version;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 79ffe317dd..a3836d315a 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -512,6 +513,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/collationcmds.c b/src/backend/commands/collationcmds.c
index 5ad8886e60..519f7a7df3 100644
--- a/src/backend/commands/collationcmds.c
+++ b/src/backend/commands/collationcmds.c
@@ -270,28 +270,12 @@ Datum
pg_collation_actual_version(PG_FUNCTION_ARGS)
{
Oid collid = PG_GETARG_OID(0);
- HeapTuple tp;
- char *collcollate;
- char collprovider;
char *version;
- tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(collid));
- if (!HeapTupleIsValid(tp))
- ereport(ERROR,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("collation with OID %u does not exist", collid)));
-
- collcollate = pstrdup(NameStr(((Form_pg_collation) GETSTRUCT(tp))->collcollate));
- collprovider = ((Form_pg_collation) GETSTRUCT(tp))->collprovider;
-
- ReleaseSysCache(tp);
-
- version = get_collation_actual_version(collprovider, collcollate);
+ version = get_collation_version_for_oid(collid);
+ Assert(version);
- if (version)
- PG_RETURN_TEXT_P(cstring_to_text(version));
- else
- PG_RETURN_NULL();
+ PG_RETURN_TEXT_P(cstring_to_text(version));
}
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 22228f5684..1084931753 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -632,6 +634,36 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL, as VACUUM FULL will recreate the index and
+ * update the recorded collation version.
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index 25545029d7..ad1383e7b3 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 514e0fa0af..7e5fcb77df 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -139,6 +141,9 @@ static char *IsoLocaleName(const char *); /* MSVC specific */
static void icu_set_collation_attributes(UCollator *collator, const char *loc);
#endif
+static char *get_collation_actual_version(char collprovider,
+ const char *collcollate);
+
/*
* pg_perm_setlocale
*
@@ -1630,7 +1635,7 @@ pg_newlocale_from_collation(Oid collid)
* Get provider-specific collation version string for the given collation from
* the operating system/library.
*/
-char *
+static char *
get_collation_actual_version(char collprovider, const char *collcollate)
{
char *collversion = NULL;
@@ -1712,6 +1717,51 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ *
+ * As this version is used in index dependency tracking, an empty string is
+ * returned when the version is unknown, to distinguish from dependencies where
+ * the version should not be tracked, represented by a NULL version.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version = NULL;
+
+ Assert(oid != C_COLLATION_OID && oid != POSIX_COLLATION_OID);
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ if (!version)
+ return "";
+ else
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..4ba8c8c637 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -13,6 +13,7 @@
#include "catalog/binary_upgrade.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_type.h"
#include "commands/extension.h"
@@ -197,3 +198,27 @@ binary_upgrade_set_missing_value(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+
+Datum
+binary_upgrade_set_index_coll_version(PG_FUNCTION_ARGS)
+{
+ Oid relid;
+ Oid coll;
+ char *version;
+
+ CHECK_IS_BINARY_UPGRADE;
+
+ relid = PG_GETARG_OID(0);
+
+ /* Detect if a collation is specified */
+ if (PG_ARGISNULL(1))
+ coll = InvalidOid;
+ else
+ coll = PG_GETARG_OID(1);
+
+ version = TextDatumGetCString(PG_GETARG_TEXT_PP(2));
+
+ index_force_collation_versions(relid, coll, version);
+
+ PG_RETURN_VOID();
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..a1d40bed29 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -42,6 +42,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/partition.h"
@@ -5927,6 +5928,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/bin/pg_dump/Makefile b/src/bin/pg_dump/Makefile
index 2532d9183a..c58ebaa681 100644
--- a/src/bin/pg_dump/Makefile
+++ b/src/bin/pg_dump/Makefile
@@ -16,6 +16,8 @@ subdir = src/bin/pg_dump
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
+
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..a94fbafec3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -171,6 +171,7 @@ typedef struct _dumpOptions
int sequence_data; /* dump sequence data even in schema-only mode */
int do_nothing;
+ int unknown_coll_compat;
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index af8d38488e..60feb0663a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -46,6 +46,7 @@
#include "catalog/pg_attribute_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
+#include "catalog/pg_collation_d.h"
#include "catalog/pg_default_acl_d.h"
#include "catalog/pg_largeobject_d.h"
#include "catalog/pg_largeobject_metadata_d.h"
@@ -285,6 +286,8 @@ static void binary_upgrade_extension_member(PQExpBuffer upgrade_buffer,
static const char *getAttrName(int attrnum, TableInfo *tblInfo);
static const char *fmtCopyColumnList(const TableInfo *ti, PQExpBuffer buffer);
static bool nonemptyReloptions(const char *reloptions);
+static void appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo,
+ int enc, int unknown_coll_compat);
static void appendReloptionsArrayAH(PQExpBuffer buffer, const char *reloptions,
const char *prefix, Archive *fout);
static char *get_synchronized_snapshot(Archive *fout);
@@ -386,6 +389,7 @@ main(int argc, char **argv)
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
{"include-foreign-data", required_argument, NULL, 11},
+ {"unknown-collations-binary-compatible", no_argument, &dopt.unknown_coll_compat, 1},
{NULL, 0, NULL, 0}
};
@@ -713,6 +717,10 @@ main(int argc, char **argv)
if (archiveFormat != archDirectory && numWorkers > 1)
fatal("parallel backup only supported by the directory format");
+ /* Unknown collation versions can only be ignored in binary upgrade mode */
+ if (dopt.unknown_coll_compat && !dopt.binary_upgrade)
+ fatal("option --unknown-collations-binary-compatible only works in binary upgrade mode");
+
/* Open the output file */
fout = CreateArchive(filename, archiveFormat, compressLevel, dosync,
archiveMode, setupDumpWorker);
@@ -7017,7 +7025,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_tablespace,
i_indreloptions,
i_indstatcols,
- i_indstatvals;
+ i_indstatvals,
+ i_inddependoids,
+ i_inddependversions;
int ntups;
for (i = 0; i < numTables; i++)
@@ -7053,7 +7063,62 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
* is not.
*/
resetPQExpBuffer(query);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 140000)
+ {
+ appendPQExpBuffer(query,
+ "SELECT t.tableoid, t.oid, "
+ "t.relname AS indexname, "
+ "inh.inhparent AS parentidx, "
+ "pg_catalog.pg_get_indexdef(i.indexrelid) AS indexdef, "
+ "i.indnkeyatts AS indnkeyatts, "
+ "i.indnatts AS indnatts, "
+ "i.indkey, i.indisclustered, "
+ "i.indisreplident, "
+ "c.contype, c.conname, "
+ "c.condeferrable, c.condeferred, "
+ "c.tableoid AS contableoid, "
+ "c.oid AS conoid, "
+ "pg_catalog.pg_get_constraintdef(c.oid, false) AS condef, "
+ "(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
+ "t.reloptions AS indreloptions, "
+ "(SELECT pg_catalog.array_agg(attnum ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatcols,"
+ "(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
+ " FROM pg_catalog.pg_attribute "
+ " WHERE attrelid = i.indexrelid AND "
+ " attstattarget >= 0) AS indstatvals, "
+ "(SELECT pg_catalog.array_agg(refobjid ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependoids, "
+ "(SELECT pg_catalog.array_agg(quote_literal(refobjversion) ORDER BY refobjid) "
+ " FROM pg_catalog.pg_depend "
+ " WHERE classid = " CppAsString2(RelationRelationId) " AND "
+ " objid = i.indexrelid AND "
+ " objsubid = 0 AND "
+ " refclassid = " CppAsString2(CollationRelationId) " AND "
+ " refobjversion IS NOT NULL) AS inddependversions "
+ "FROM pg_catalog.pg_index i "
+ "JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
+ "JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
+ "LEFT JOIN pg_catalog.pg_constraint c "
+ "ON (i.indrelid = c.conrelid AND "
+ "i.indexrelid = c.conindid AND "
+ "c.contype IN ('p','u','x')) "
+ "LEFT JOIN pg_catalog.pg_inherits inh "
+ "ON (inh.inhrelid = indexrelid) "
+ "WHERE i.indrelid = '%u'::pg_catalog.oid "
+ "AND (i.indisvalid OR t2.relkind = 'p') "
+ "AND i.indisready "
+ "ORDER BY indexname",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
appendPQExpBuffer(query,
"SELECT t.tableoid, t.oid, "
@@ -7078,7 +7143,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT pg_catalog.array_agg(attstattarget ORDER BY attnum) "
" FROM pg_catalog.pg_attribute "
" WHERE attrelid = i.indexrelid AND "
- " attstattarget >= 0) AS indstatvals "
+ " attstattarget >= 0) AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"JOIN pg_catalog.pg_class t2 ON (t2.oid = i.indrelid) "
@@ -7117,7 +7184,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7152,7 +7221,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_constraint c "
@@ -7183,7 +7254,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"t.reloptions AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7217,7 +7290,9 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
"(SELECT spcname FROM pg_catalog.pg_tablespace s WHERE s.oid = t.reltablespace) AS tablespace, "
"null AS indreloptions, "
"'' AS indstatcols, "
- "'' AS indstatvals "
+ "'' AS indstatvals, "
+ "' ' AS inddependoids, "
+ "' ' AS inddependversions "
"FROM pg_catalog.pg_index i "
"JOIN pg_catalog.pg_class t ON (t.oid = i.indexrelid) "
"LEFT JOIN pg_catalog.pg_depend d "
@@ -7257,6 +7332,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
i_indreloptions = PQfnumber(res, "indreloptions");
i_indstatcols = PQfnumber(res, "indstatcols");
i_indstatvals = PQfnumber(res, "indstatvals");
+ i_inddependoids = PQfnumber(res, "inddependoids");
+ i_inddependversions = PQfnumber(res, "inddependversions");
tbinfo->indexes = indxinfo =
(IndxInfo *) pg_malloc(ntups * sizeof(IndxInfo));
@@ -7282,6 +7359,8 @@ getIndexes(Archive *fout, TableInfo tblinfo[], int numTables)
indxinfo[j].indreloptions = pg_strdup(PQgetvalue(res, j, i_indreloptions));
indxinfo[j].indstatcols = pg_strdup(PQgetvalue(res, j, i_indstatcols));
indxinfo[j].indstatvals = pg_strdup(PQgetvalue(res, j, i_indstatvals));
+ indxinfo[j].inddependoids = pg_strdup(PQgetvalue(res, j, i_inddependoids));
+ indxinfo[j].inddependversions = pg_strdup(PQgetvalue(res, j, i_inddependversions));
indxinfo[j].indkeys = (Oid *) pg_malloc(indxinfo[j].indnattrs * sizeof(Oid));
parseOidArray(PQgetvalue(res, j, i_indkey),
indxinfo[j].indkeys, indxinfo[j].indnattrs);
@@ -16319,7 +16398,8 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
/*
* If there's an associated constraint, don't dump the index per se, but
- * do dump any comment for it. (This is safe because dependency ordering
+ * do dump any comment, or in binary upgrade mode dependency on a
+ * collation version for it. (This is safe because dependency ordering
* will have ensured the constraint is emitted first.) Note that the
* emitted comment has to be shown as depending on the constraint, not the
* index, in such cases.
@@ -16386,6 +16466,10 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
"pg_catalog.pg_class",
"INDEX", qqindxname);
+ if (dopt->binary_upgrade)
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
/* If the index defines identity, we need to record that. */
if (indxinfo->indisreplident)
{
@@ -16414,6 +16498,21 @@ dumpIndex(Archive *fout, IndxInfo *indxinfo)
if (indstatvalsarray)
free(indstatvalsarray);
}
+ else if (dopt->binary_upgrade)
+ {
+ appendIndexCollationVersion(q, indxinfo, fout->encoding,
+ dopt->unknown_coll_compat);
+
+ if (indxinfo->dobj.dump & DUMP_COMPONENT_DEFINITION)
+ ArchiveEntry(fout, indxinfo->dobj.catId, indxinfo->dobj.dumpId,
+ ARCHIVE_OPTS(.tag = indxinfo->dobj.name,
+ .namespace = tbinfo->dobj.namespace->dobj.name,
+ .tablespace = indxinfo->tablespace,
+ .owner = tbinfo->rolname,
+ .description = "INDEX",
+ .section = SECTION_POST_DATA,
+ .createStmt = q->data));
+ }
/* Dump Index Comments */
if (indxinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
@@ -18398,6 +18497,77 @@ nonemptyReloptions(const char *reloptions)
return (reloptions != NULL && strlen(reloptions) > 2);
}
+/*
+ * Format inddependoids and inddependversions arrays and append it to the given
+ * buffer in the form of binary_upgrade_set_index_coll_version() calls.
+ */
+static void
+appendIndexCollationVersion(PQExpBuffer buffer, IndxInfo *indxinfo, int enc,
+ int unknown_coll_compat)
+{
+ char *inddependoids = indxinfo->inddependoids;
+ char *inddependversions = indxinfo->inddependversions;
+ char **inddependoidsarray = NULL;
+ char **inddependversionsarray = NULL;
+ int ninddependoids;
+ int ninddependversions;
+ int i;
+
+ /*
+ * for older versions that don't record the collation depndency, issue a
+ * statement to mark the collation version as unknown
+ */
+ if (strcmp(inddependoids, " ") == 0)
+ {
+ /*
+ * do not issue UNKNOWN VERSION if caller specified that those are
+ * compatible
+ */
+ if (unknown_coll_compat)
+ return;
+
+ Assert(strcmp(inddependversions, " ") == 0);
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, NULL, '');",
+ indxinfo->dobj.catId.oid);
+ return;
+ }
+
+ parsePGArray(inddependoids, &inddependoidsarray, &ninddependoids);
+ parsePGArray(inddependversions, &inddependversionsarray, &ninddependversions);
+
+ Assert(ninddependoids == ninddependversions);
+
+ for (i = 0; i < ninddependoids; i++)
+ {
+ /*
+ * If there was an unknown version dependency recorded for this
+ * collation and the caller asked to mark those as depending on the
+ * current version, don't emit a binary_upgrade_set_index_coll_version
+ * function call.
+ */
+ if ((strcmp(inddependversionsarray[i], "''")) == 0
+ && unknown_coll_compat)
+ {
+ continue;
+ }
+
+ appendPQExpBufferStr(buffer, "\n-- For binary upgrade, restore dependent collation version.\n");
+ appendPQExpBuffer(buffer, "SELECT "
+ "pg_catalog.binary_upgrade_set_index_coll_version(%d, %s, %s);",
+ indxinfo->dobj.catId.oid,
+ inddependoidsarray[i],
+ inddependversionsarray[i]);
+ }
+
+ if (inddependoidsarray)
+ free(inddependoidsarray);
+ if (inddependversionsarray)
+ free(inddependversionsarray);
+}
+
/*
* Format a reloptions array and append it to the given buffer.
*
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..6c7d1fe158 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -366,6 +366,9 @@ typedef struct _indxInfo
int indnattrs; /* total number of index attributes */
Oid *indkeys; /* In spite of the name 'indkeys' this field
* contains both key and nonkey attributes */
+ char *inddependoids; /* oids of collation this index depends on */
+ char *inddependversions; /* version of collation this index depends
+ * on */
bool indisclustered;
bool indisreplident;
Oid parentidx; /* if partitioned, parent index OID */
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..5f323efb1f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -53,6 +53,24 @@ my %pgdump_runs = (
"$tempdir/binary_upgrade.dump",
],
},
+ binary_coll_compatible => {
+ dump_cmd => [
+ 'pg_dump',
+ '--no-sync',
+ '--format=custom',
+ "--file=$tempdir/binary_coll_compatible.dump",
+ '-w',
+ '--schema-only',
+ '--binary-upgrade',
+ '--unknown-collations-binary-compatible',
+ '-d', 'postgres', # alternative way to specify database
+ ],
+ restore_cmd => [
+ 'pg_restore', '-Fc', '--verbose',
+ "--file=$tempdir/binary_coll_compatible.sql",
+ "$tempdir/binary_coll_compatible.dump",
+ ],
+ },
clean => {
dump_cmd => [
'pg_dump',
@@ -387,6 +405,7 @@ my %dump_test_schema_runs = (
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -920,9 +939,10 @@ my %tests = (
test_schema_plus_blobs => 1,
},
unlike => {
- binary_upgrade => 1,
- no_blobs => 1,
- schema_only => 1,
+ binary_upgrade => 1,
+ binary_coll_compatible => 1,
+ no_blobs => 1,
+ schema_only => 1,
},
},
@@ -1184,6 +1204,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
@@ -1209,6 +1230,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1244,6 +1266,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1266,6 +1289,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1287,6 +1311,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1308,6 +1333,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -1674,6 +1700,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -1688,7 +1715,7 @@ my %tests = (
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
\n/xms,
- like => { binary_upgrade => 1, },
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
},
'CREATE TYPE dump_test.textrange AS RANGE' => {
@@ -2356,6 +2383,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2549,6 +2577,7 @@ my %tests = (
},
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
},
},
@@ -2617,6 +2646,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -2688,6 +2718,7 @@ my %tests = (
/xm,
like => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
@@ -3155,6 +3186,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3170,6 +3202,7 @@ my %tests = (
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => {
binary_upgrade => 1,
+ binary_coll_compatible => 1,
exclude_dump_test_schema => 1,
schema_only => 1,
},
@@ -3302,16 +3335,53 @@ my %tests = (
%full_runs, %dump_test_schema_runs, section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1 },
+ },
+
+ "binary_upgrade_set_index_coll_version(oid, oid, 'not_a_version')" => {
+ create_order => 101,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll(id integer, val text);
+ CREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'not_a_version\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_idx1\';',
+ regexp => qr/^
+ \QCREATE INDEX regress_coll_idx1 ON dump_test.regress_table_coll USING btree (val COLLATE "fr-x-icu");\E\n
+ \n
+ \Q-- For binary upgrade, restore dependent collation version.\E\n
+ \QSELECT pg_catalog.binary_upgrade_set_index_coll_version\E \(\d+,\ \d+,\ 'not_a_version'\);/xm,
+ like => { binary_upgrade => 1, binary_coll_compatible => 1, },
+ icu => 1,
+ },
+ "binary_upgrade_set_index_coll_version(?, ?, '')" => {
+ create_order => 102,
+ create_sql => '
+ CREATE TABLE dump_test.regress_table_coll_no_ver(id integer, val text);
+ CREATE INDEX regress_coll_no_ver_idx1 ON dump_test.regress_table_coll_no_ver(val COLLATE "fr-x-icu");
+ UPDATE pg_depend SET refobjversion = \'\' WHERE refobjversion IS NOT NULL AND objid::regclass::text = \'dump_test.regress_coll_no_ver_idx1\';',
+ regexp => qr/SELECT pg_catalog.binary_upgrade_set_index_coll_version\(\d+, \d+, ''\)/,
+ like => { binary_upgrade => 1},
+ # should not appear in binary_coll_compatible case!
+ unlike => { binary_coll_compatible => 1},
+ icu => 1,
});
#########################################
# Create a PG instance to test actually dumping from
-my $node = get_new_node('main');
-$node->init;
-$node->start;
+my $main_node = get_new_node('main');
+$main_node->init;
+$main_node->start;
+
+my $port = $main_node->port;
+
+# And another instance to validate the binary dump
+my $bin_node = get_new_node('binary');
+$bin_node->init;
+$bin_node->start;
-my $port = $node->port;
+my $bin_port = $bin_node->port;
+
+# and add a $node variable pointing to main_node for now
+my $node = $main_node;
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
@@ -3335,6 +3405,10 @@ $node->psql('postgres', 'create database regress_pg_dump_test;');
# command_fails_like is actually 2 tests)
my $num_tests = 12;
+# 4 more tests for restoring globals and binary_upgrade dump, dumping it again
+# and regenerating the sql file
+$num_tests+= 4;
+
foreach my $run (sort keys %pgdump_runs)
{
my $test_key = $run;
@@ -3385,16 +3459,29 @@ foreach my $run (sort keys %pgdump_runs)
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# If there is a like entry, but no unlike entry, then we will test the like case
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
{
$num_tests++;
+
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
else
{
# We will test everything that isn't a 'like'
$num_tests++;
+ # binary_upgrade tests are also run after being restored and
+ # re-dumped.
+ $num_tests++ if ($test_key eq 'binary_upgrade');
}
}
}
@@ -3442,6 +3529,12 @@ foreach my $test (
next;
}
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
# Add terminating semicolon
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
}
@@ -3495,79 +3588,116 @@ command_fails_like(
#########################################
# Run all runs
-foreach my $run (sort keys %pgdump_runs)
+foreach my $pass (1, 2)
{
- my $test_key = $run;
- my $run_db = 'postgres';
-
- $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
- "$run: pg_dump runs");
-
- if ($pgdump_runs{$run}->{restore_cmd})
+ foreach my $run (sort keys %pgdump_runs)
{
- $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
- "$run: pg_restore runs");
- }
-
- if ($pgdump_runs{$run}->{test_key})
- {
- $test_key = $pgdump_runs{$run}->{test_key};
- }
-
- my $output_file = slurp_file("$tempdir/${run}.sql");
+ my $test_key = $run;
+ my $run_db = 'postgres';
- #########################################
- # Run all tests where this run is included
- # as either a 'like' or 'unlike' test.
+ # we only test binary upgrade on the 2nd pass
+ next if ($pass == 2 and $test_key ne 'binary_upgrade');
- foreach my $test (sort keys %tests)
- {
- my $test_db = 'postgres';
+ $node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
+ "$run: pg_dump runs");
- if (defined($pgdump_runs{$run}->{database}))
+ if ($pgdump_runs{$run}->{restore_cmd})
{
- $run_db = $pgdump_runs{$run}->{database};
+ $node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
+ "$run: pg_restore runs");
}
- if (defined($tests{$test}->{database}))
+ if ($pgdump_runs{$run}->{test_key})
{
- $test_db = $tests{$test}->{database};
+ $test_key = $pgdump_runs{$run}->{test_key};
}
- # Skip any collation-related commands if there is no collation support
- if (!$collation_support && defined($tests{$test}->{collation}))
- {
- next;
- }
+ my $output_file = slurp_file("$tempdir/${run}.sql");
- if ($run_db ne $test_db)
- {
- next;
- }
+ #########################################
+ # Run all tests where this run is included
+ # as either a 'like' or 'unlike' test.
- # Run the test listed as a like, unless it is specifically noted
- # as an unlike (generally due to an explicit exclusion or similar).
- if ($tests{$test}->{like}->{$test_key}
- && !defined($tests{$test}->{unlike}->{$test_key}))
+ foreach my $test (sort keys %tests)
{
- if (!ok($output_file =~ $tests{$test}->{regexp},
- "$run: should dump $test"))
+ my $test_db = 'postgres';
+
+ if (defined($pgdump_runs{$run}->{database}))
{
- diag("Review $run results in $tempdir");
+ $run_db = $pgdump_runs{$run}->{database};
}
- }
- else
- {
- if (!ok($output_file !~ $tests{$test}->{regexp},
- "$run: should not dump $test"))
+
+ if (defined($tests{$test}->{database}))
+ {
+ $test_db = $tests{$test}->{database};
+ }
+
+ # Skip any collation-related commands if there is no collation support
+ if (!$collation_support && defined($tests{$test}->{collation}))
+ {
+ next;
+ }
+
+ # Skip any icu-related commands if there is no icu support
+ if ($ENV{with_icu} ne 'yes' && defined($tests{$test}->{icu}))
+ {
+ next;
+ }
+
+ if ($run_db ne $test_db)
+ {
+ next;
+ }
+
+ # Run the test listed as a like, unless it is specifically noted
+ # as an unlike (generally due to an explicit exclusion or similar).
+ if ($tests{$test}->{like}->{$test_key}
+ && !defined($tests{$test}->{unlike}->{$test_key}))
{
- diag("Review $run results in $tempdir");
+ if (!ok($output_file =~ $tests{$test}->{regexp},
+ "$run: should dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
+ }
+ else
+ {
+ if (!ok($output_file !~ $tests{$test}->{regexp},
+ "$run: should not dump $test"))
+ {
+ diag("Review $run results in $tempdir");
+ }
}
}
}
+
+ # After all dump have been generated, restore the binary_upgrade dump with
+ # the required global objects on a suitable node, and continue with the 2nd
+ # pass.
+ if ($pass == 1)
+ {
+ # Stop the original database instance as we don't need it anymore.
+ $node->stop('fast');
+
+ $bin_node->command_ok(\@{['psql',
+ "-d", "postgres", "-f", "$tempdir/pg_dumpall_globals.sql"]},
+ "Restore globals");
+
+ $bin_node->stop('fast');
+ $bin_node->start(binary_start => 1);
+ $bin_node->command_ok(\@{['pg_restore', '-p', $bin_port,
+ '-d', 'postgres',
+ "$tempdir/binary_upgrade.dump"]},
+ "Restore the binary_upgrade dump");
+ $bin_node->stop('fast');
+ $bin_node->start;
+
+ # And change $node to point to the freshly restored node.
+ $node = $bin_node;
+ }
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
-$node->stop('fast');
+$bin_node->stop('fast');
diff --git a/src/bin/pg_upgrade/dump.c b/src/bin/pg_upgrade/dump.c
index 4d730adfe2..672ecda169 100644
--- a/src/bin/pg_upgrade/dump.c
+++ b/src/bin/pg_upgrade/dump.c
@@ -52,9 +52,11 @@ generate_old_dump(void)
parallel_exec_prog(log_file_name, NULL,
"\"%s/pg_dump\" %s --schema-only --quote-all-identifiers "
- "--binary-upgrade --format=custom %s --file=\"%s\" %s",
+ "--binary-upgrade --format=custom %s %s --file=\"%s\" %s",
new_cluster.bindir, cluster_conn_opts(&old_cluster),
log_opts.verbose ? "--verbose" : "",
+ user_opts.coll_compat ?
+ "--unknown-collations-binary-compatible" : "",
sql_file_name, escaped_connstr.data);
termPQExpBuffer(&escaped_connstr);
diff --git a/src/bin/pg_upgrade/option.c b/src/bin/pg_upgrade/option.c
index aca1ee8b48..c7e291f7e7 100644
--- a/src/bin/pg_upgrade/option.c
+++ b/src/bin/pg_upgrade/option.c
@@ -56,6 +56,7 @@ parseCommandLine(int argc, char *argv[])
{"socketdir", required_argument, NULL, 's'},
{"verbose", no_argument, NULL, 'v'},
{"clone", no_argument, NULL, 1},
+ {"collation-binary-compatible", no_argument, NULL, 2},
{NULL, 0, NULL, 0}
};
@@ -203,6 +204,10 @@ parseCommandLine(int argc, char *argv[])
user_opts.transfer_mode = TRANSFER_MODE_CLONE;
break;
+ case 2:
+ user_opts.coll_compat = true;
+ break;
+
default:
fprintf(stderr, _("Try \"%s --help\" for more information.\n"),
os_info.progname);
@@ -307,6 +312,8 @@ usage(void)
printf(_(" -v, --verbose enable verbose internal logging\n"));
printf(_(" -V, --version display version information, then exit\n"));
printf(_(" --clone clone instead of copying files to new cluster\n"));
+ printf(_(" --collation-binary-compatible mark collations as depending on current collation\n"
+ " versions rather than unknown if they're unknown\n"));
printf(_(" -?, --help show this help, then exit\n"));
printf(_("\n"
"Before running pg_upgrade you must:\n"
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
index 8b90cefbe0..68e637ce8a 100644
--- a/src/bin/pg_upgrade/pg_upgrade.h
+++ b/src/bin/pg_upgrade/pg_upgrade.h
@@ -292,6 +292,8 @@ typedef struct
transferMode transfer_mode; /* copy files or link them? */
int jobs; /* number of processes/threads to use */
char *socketdir; /* directory to use for Unix sockets */
+ bool coll_compat; /* should we skip marking index collations as
+ * unknown version */
} UserOpts;
typedef struct
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 3baa5e498a..2966339498 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -130,6 +130,13 @@ typedef enum ObjectClass
#define LAST_OCLASS OCLASS_TRANSFORM
+/* Struct describing one forced collation version dependency */
+typedef struct NewCollationVersionDependency
+{
+ char *version; /* forced collation version */
+ Oid oid; /* target collation oid */
+} NewCollationVersionDependency;
+
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
#define PERFORM_DELETION_CONCURRENTLY 0x0002 /* concurrent drop */
@@ -160,7 +167,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -180,17 +188,29 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations,
+ bool record_version);
+
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
int nreferenced,
- const char *version,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
@@ -209,10 +229,9 @@ extern long changeDependencyFor(Oid classId, Oid objectId,
Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
-extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
+long changeDependenciesOf(Oid classId, Oid oldObjectId,
Oid newObjectId);
-
-extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
+long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index f58e8675f3..9b4de26514 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,11 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
+extern void index_force_collation_versions(Oid indexid, Oid coll,
+ char *version);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
@@ -131,6 +136,8 @@ extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
extern void index_set_state_flags(Oid indexId, IndexStateFlagsAction action);
+extern void index_update_collation_versions(Oid relid);
+
extern Oid IndexGetRelation(Oid indexId, bool missing_ok);
extern void reindex_index(Oid indexId, bool skip_constraint_checks,
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d45469c8b4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10349,6 +10349,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '8178', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_index_coll_version', provolatile => 'v',
+ proparallel => 'u', prorettype => 'void', proargtypes => 'oid oid text',
+ prosrc => 'binary_upgrade_set_index_coll_version' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 7b37562648..2d511c5cba 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -361,6 +361,8 @@ extern void GenerateTypeDependencies(HeapTuple typeTuple,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..96da132c03 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -103,7 +103,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
-extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 0b5957ba02..f79667d651 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*----------
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/perl/PostgresNode.pm b/src/test/perl/PostgresNode.pm
index 8c1b77376f..e014c4f9c6 100644
--- a/src/test/perl/PostgresNode.pm
+++ b/src/test/perl/PostgresNode.pm
@@ -765,10 +765,14 @@ sub start
local %ENV = %ENV;
delete $ENV{PGAPPNAME};
+ my $options = "--cluster-name=$name";
+
+ $options .= ' -b' if ($params{binary_start});
+
# Note: We set the cluster_name here, not in postgresql.conf (in
# sub init) so that it does not get copied to standbys.
$ret = TestLib::system_log('pg_ctl', '-D', $self->data_dir, '-l',
- $self->logfile, '-o', "--cluster-name=$name", 'start');
+ $self->logfile, '-o', $options, 'start');
}
if ($ret != 0)
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..0075e85072 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,163 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "es-x-icu" | up to date
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | up to date
+ icuidx02_d_en_fr | "en-x-icu" | up to date
+ icuidx02_d_en_fr | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "en-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "fr-x-icu" | up to date
+ icuidx03_t_en_fr_ga | "ga-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx04_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | up to date
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "default" | up to date
+ icuidx06_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx06_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "default" | up to date
+ icuidx07_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx07_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx08_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "en-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "fr-x-icu" | up to date
+ icuidx09_d_en_fr_ga | "ga-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | up to date
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | up to date
+ icuidx11_d_es | "default" | up to date
+ icuidx11_d_es | "es-x-icu" | up to date
+ icuidx12_custom | "default" | up to date
+ icuidx12_custom | custom | up to date
+ icuidx13_custom | "default" | up to date
+ icuidx13_custom | custom | up to date
+ icuidx14_myrange | "default" | up to date
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | up to date
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | up to date
+ icuidx16_mood | "fr-x-icu" | up to date
+ icuidx17_part | "en-x-icu" | up to date
+ icuidx18_hash_d_es | "es-x-icu" | up to date
+ icuidx19_hash_id_d_es_eq | "default" | up to date
+ icuidx19_hash_id_d_es_eq | "es-x-icu" | up to date
+ icuidx20_hash_id_d_es_lt | "default" | up to date
+ icuidx20_hash_id_d_es_lt | "es-x-icu" | up to date
+(49 rows)
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..ce3d45cf2c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..e93530af55 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,112 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- for key columns, hash indexes should record dependency on the collation but
+-- not the version
+CREATE INDEX icuidx18_hash_d_es ON collate_test USING hash (d_es);
+CREATE INDEX icuidx19_hash_id_d_es_eq ON collate_test USING hash (id) WHERE (d_es) = 'foo';
+CREATE INDEX icuidx20_hash_id_d_es_lt ON collate_test USING hash (id) WHERE (d_es) < 'foo';
+
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'version not tracked'
+ELSE CASE
+ WHEN refobjversion = pg_collation_actual_version(refobjid) THEN 'up to date'
+ ELSE 'out of date'
+ END
+END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+-- Validate that REINDEX will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that REINDEX CONCURRENTLY will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE CONCURRENTLY collate_test;
+REINDEX TABLE CONCURRENTLY collate_part_0;
+REINDEX INDEX CONCURRENTLY icuidx17_part;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+
+-- Validate that VACUUM FULL will update the stored version.
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+VACUUM FULL collate_test;
+VACUUM FULL collate_part_0;
+VACUUM FULL collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--gclmslubawp3yaq7
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename="v28-0004-Add-ALTER-INDEX-.-ALTER-COLLATION-.-REFRESH-VERS.patch"
^ permalink raw reply [nested|flat] 98+ messages in thread
* [PATCH 4/6] Track collation versions for indexes.
@ 2019-05-28 18:16 Thomas Munro <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Thomas Munro @ 2019-05-28 18:16 UTC (permalink / raw)
Record the current version of dependent collations in pg_depend when
creating or rebuilding an index. That version is checked against current
version whenever we call get_relation_info for an index or open the parent
table during non-full VACUUM or ANALYZE. Warn that the index may be corrupted
if the versions don't match.
A new flag is added in RelationData to specify that the check has already beed
done to avoid checking and reporting the message multiple time in a backend
lifetime.
Author: Thomas Munro, Julien Rouhaud
Reviewed-by:
Discussion: https://postgr.es/m/CAEepm%3D0uEQCpfq_%2BLYFBdArCe4Ot98t1aR4eYiYTe%3DyavQygiQ%40mail.gmail.com
---
src/backend/catalog/dependency.c | 189 ++++++++++++++++--
src/backend/catalog/heap.c | 7 +-
src/backend/catalog/index.c | 143 ++++++++++++-
src/backend/catalog/pg_constraint.c | 2 +-
src/backend/catalog/pg_depend.c | 127 ++++++++++--
src/backend/catalog/pg_type.c | 69 +++++++
src/backend/commands/vacuum.c | 31 +++
src/backend/optimizer/util/plancat.c | 9 +
src/backend/utils/adt/pg_locale.c | 38 ++++
src/backend/utils/cache/relcache.c | 2 +
src/include/catalog/dependency.h | 21 +-
src/include/catalog/index.h | 2 +
src/include/catalog/pg_type.h | 2 +
src/include/utils/pg_locale.h | 1 +
src/include/utils/rel.h | 1 +
src/test/Makefile | 5 +-
src/test/locale/.gitignore | 1 +
src/test/locale/Makefile | 7 +
src/test/locale/t/001_index.pl | 76 +++++++
.../regress/expected/collate.icu.utf8.out | 118 +++++++++++
...te.icu.utf8.out => collate.icu.utf8_2.out} | 118 +++++++++++
src/test/regress/expected/create_index.out | 8 +-
src/test/regress/sql/collate.icu.utf8.sql | 76 +++++++
23 files changed, 992 insertions(+), 61 deletions(-)
create mode 100644 src/test/locale/t/001_index.pl
copy src/test/regress/expected/{collate.icu.utf8.out => collate.icu.utf8_2.out} (89%)
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 78c31baa34..93f57cd633 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -77,6 +77,7 @@
#include "parser/parsetree.h"
#include "rewrite/rewriteRemove.h"
#include "storage/lmgr.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
#include "utils/lsyscache.h"
@@ -137,6 +138,9 @@ typedef struct
{
ObjectAddresses *addrs; /* addresses being accumulated */
List *rtables; /* list of rangetables to resolve Vars */
+ bool track_version; /* whether caller asked to track dependency
+ * versions */
+ NodeTag type; /* nodetag of the current node */
} find_expr_references_context;
/*
@@ -437,6 +441,80 @@ performMultipleDeletions(const ObjectAddresses *objects,
table_close(depRel, RowExclusiveLock);
}
+/*
+ * Call a function for all objects that depend on 'object'. If the function
+ * returns a non-NULL pointer to a new version string, update the version.
+ */
+void
+visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ ObjectAddress otherObject;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(object->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(object->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+ char *cur_version, *new_version;
+ Datum depversion;
+ bool isnull;
+
+ otherObject.classId = foundDep->refclassid;
+ otherObject.objectId = foundDep->refobjid;
+ otherObject.objectSubId = foundDep->refobjsubid;
+
+ depversion = heap_getattr(tup, Anum_pg_depend_refobjversion,
+ RelationGetDescr(depRel), &isnull);
+
+ cur_version = isnull ? NULL : TextDatumGetCString(depversion);
+
+ new_version = callback(&otherObject, cur_version, userdata);
+ if (new_version)
+ {
+ Datum values[Natts_pg_depend];
+ bool nulls[Natts_pg_depend];
+ bool replaces[Natts_pg_depend];
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replaces, false, sizeof(replaces));
+
+ values[Anum_pg_depend_refobjversion - 1] =
+ CStringGetTextDatum(new_version);
+ replaces[Anum_pg_depend_refobjversion - 1] = true;
+
+ tup = heap_modify_tuple(tup, RelationGetDescr(depRel), values,
+ nulls, replaces);
+ CatalogTupleUpdate(depRel, &tup->t_self, tup);
+
+ heap_freetuple(tup);
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+}
+
/*
* findDependentObjects - find all objects that depend on 'object'
*
@@ -1590,6 +1668,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
find_expr_references_context context;
context.addrs = new_object_addresses();
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* Set up interpretation for Vars at varlevelsup = 0 */
context.rtables = list_make1(rtable);
@@ -1602,9 +1684,10 @@ recordDependencyOnExpr(const ObjectAddress *depender,
/* And record 'em */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ false);
free_object_addresses(context.addrs);
}
@@ -1631,12 +1714,18 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self)
+ bool reverse_self,
+ bool track_version)
{
find_expr_references_context context;
RangeTblEntry rte;
context.addrs = new_object_addresses();
+ context.track_version = track_version;
+ if (expr)
+ context.type = expr->type;
+ else
+ context.type = T_Invalid;
/* We gin up a rather bogus rangetable list to handle Vars */
MemSet(&rte, 0, sizeof(rte));
@@ -1690,9 +1779,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the self-dependencies with the appropriate direction */
if (!reverse_self)
recordMultipleDependencies(depender,
- self_addrs->refs, NULL,
+ self_addrs->refs,
self_addrs->numrefs,
- self_behavior);
+ self_behavior,
+ track_version);
else
{
/* Can't use recordMultipleDependencies, so do it the hard way */
@@ -1711,9 +1801,10 @@ recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
/* Record the external dependencies */
recordMultipleDependencies(depender,
- context.addrs->refs, NULL,
+ context.addrs->refs,
context.addrs->numrefs,
- behavior);
+ behavior,
+ track_version);
free_object_addresses(context.addrs);
}
@@ -1735,8 +1826,13 @@ static bool
find_expr_references_walker(Node *node,
find_expr_references_context *context)
{
+ NodeTag parent_type = context->type;
+
if (node == NULL)
return false;
+
+ context->type = node->type;
+
if (IsA(node, Var))
{
Var *var = (Var *) node;
@@ -1769,6 +1865,46 @@ find_expr_references_walker(Node *node,
/* If it's a plain relation, reference this column */
add_object_address(OCLASS_CLASS, rte->relid, var->varattno,
context->addrs);
+
+ /* Record collations from the type itself, or underlying in case of
+ * complex type. Note that if the direct parent is a CollateExpr
+ * node, there's no need to record the type underlying collation if
+ * any. A dependency already exists for the owning relation, and a
+ * change in the collation sort order wouldn't cause any harm as
+ * the collation isn't used at all in such case.
+ */
+ if (parent_type != T_CollateExpr)
+ {
+ /* type's collation if valid */
+ if (OidIsValid(var->varcollid))
+ {
+ add_object_address(OCLASS_COLLATION, var->varcollid, 0,
+ context->addrs);
+ }
+ /*
+ * otherwise, it may be a composite type having underlying
+ * collations */
+ else if (var->vartype >= FirstNormalObjectId)
+ {
+ List *collations = NIL;
+ ListCell *lc;
+
+ collations = GetTypeCollations(var->vartype, false);
+
+ foreach(lc, collations)
+ {
+ Oid coll = lfirst_oid(lc);
+
+ if (OidIsValid(coll) &&
+ (coll != DEFAULT_COLLATION_OID ||
+ context->track_version)
+ )
+ add_object_address(OCLASS_COLLATION,
+ lfirst_oid(lc), 0,
+ context->addrs);
+ }
+ }
+ }
}
/*
@@ -1793,11 +1929,13 @@ find_expr_references_walker(Node *node,
/*
* We must also depend on the constant's collation: it could be
* different from the datatype's, if a CollateExpr was const-folded to
- * a simple constant. However we can save work in the most common
- * case where the collation is "default", since we know that's pinned.
+ * a simple constant. However we can save work in the most common case
+ * where the collation is "default", since we know that's pinned, if
+ * the caller didn't ask to track dependency versions.
*/
if (OidIsValid(con->constcollid) &&
- con->constcollid != DEFAULT_COLLATION_OID)
+ (con->constcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, con->constcollid, 0,
context->addrs);
@@ -1887,7 +2025,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
- param->paramcollid != DEFAULT_COLLATION_OID)
+ (param->paramcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, param->paramcollid, 0,
context->addrs);
}
@@ -1975,7 +2114,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
- fselect->resultcollid != DEFAULT_COLLATION_OID)
+ (fselect->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, fselect->resultcollid, 0,
context->addrs);
}
@@ -2006,7 +2146,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
- relab->resultcollid != DEFAULT_COLLATION_OID)
+ (relab->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, relab->resultcollid, 0,
context->addrs);
}
@@ -2019,7 +2160,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
- iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (iocoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, iocoerce->resultcollid, 0,
context->addrs);
}
@@ -2032,7 +2174,8 @@ find_expr_references_walker(Node *node,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
- acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ (acoerce->resultcollid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, acoerce->resultcollid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2121,7 +2264,8 @@ find_expr_references_walker(Node *node,
add_object_address(OCLASS_PROC, wc->endInRangeFunc, 0,
context->addrs);
if (OidIsValid(wc->inRangeColl) &&
- wc->inRangeColl != DEFAULT_COLLATION_OID)
+ (wc->inRangeColl != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, wc->inRangeColl, 0,
context->addrs);
/* fall through to examine substructure */
@@ -2266,7 +2410,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2288,7 +2434,9 @@ find_expr_references_walker(Node *node,
{
Oid collid = lfirst_oid(ct);
- if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ if (OidIsValid(collid) &&
+ (collid != DEFAULT_COLLATION_OID ||
+ context->track_version))
add_object_address(OCLASS_COLLATION, collid, 0,
context->addrs);
}
@@ -2684,8 +2832,9 @@ record_object_address_dependencies(const ObjectAddress *depender,
{
eliminate_duplicate_dependencies(referenced);
recordMultipleDependencies(depender,
- referenced->refs, NULL, referenced->numrefs,
- behavior);
+ referenced->refs, referenced->numrefs,
+ behavior,
+ false);
}
/*
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index e31478bf91..46e9d74a97 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -2304,7 +2304,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&colobject, expr, RelationGetRelid(rel),
DEPENDENCY_AUTO,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, false);
}
else
{
@@ -2314,7 +2314,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
*/
recordDependencyOnSingleRelExpr(&defobject, expr, RelationGetRelid(rel),
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, false);
}
/*
@@ -3638,7 +3638,8 @@ StorePartitionKey(Relation rel,
RelationGetRelid(rel),
DEPENDENCY_NORMAL,
DEPENDENCY_INTERNAL,
- true /* reverse the self-deps */ );
+ true /* reverse the self-deps */,
+ false /* don't track versions */);
/*
* We must invalidate the relcache so that the next
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 8880586c37..4fb8ce76d0 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -74,6 +74,7 @@
#include "utils/guc.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/memutils.h"
#include "utils/pg_rusage.h"
#include "utils/snapmgr.h"
@@ -724,6 +725,7 @@ index_create(Relation heapRelation,
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
+ List *collations = NIL;
/* constraint flags can only be set when a constraint is requested */
Assert((constr_flags == 0) ||
@@ -1115,21 +1117,44 @@ index_create(Relation heapRelation,
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
- /* Store dependency on collations */
- /* The default collation is pinned, so don't bother recording it */
+ /*
+ * Get required distinct dependencies on collations for all index keys.
+ * Collations of directly referenced column in hash indexes can be
+ * skipped is they're deterministic.
+ */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
- if (OidIsValid(collationObjectId[i]) &&
- collationObjectId[i] != DEFAULT_COLLATION_OID)
+ Oid colloid = collationObjectId[i];
+
+ if (OidIsValid(colloid))
{
- referenced.classId = CollationRelationId;
- referenced.objectId = collationObjectId[i];
- referenced.objectSubId = 0;
+ if ((indexInfo->ii_Am != HASH_AM_OID) ||
+ !get_collation_isdeterministic(colloid))
+ collations = list_append_unique_oid(collations, colloid);
+ }
+ else
+ {
+ Form_pg_attribute att = TupleDescAttr(indexTupDesc, i);
+
+ Assert(i < indexTupDesc->natts);
- recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ collations = list_concat_unique_oid(collations,
+ GetTypeCollations(att->atttypid,
+ (indexInfo->ii_Am == HASH_AM_OID)));
}
}
+ if (collations)
+ {
+ recordDependencyOnCollations(&myself, collations);
+ /*
+ * Advance the command counter so that later calls to
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
+ }
+
/* Store dependency on operator classes */
for (i = 0; i < indexInfo->ii_NumIndexKeyAttrs; i++)
{
@@ -1143,21 +1168,29 @@ index_create(Relation heapRelation,
/* Store dependencies on anything mentioned in index expressions */
if (indexInfo->ii_Expressions)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Expressions,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
+ /*
+ * Advance the command counter so that later
+ * recordMultipleDependencies calls can see the newly-entered
+ * pg_depend catalog tuples for the index.
+ */
+ CommandCounterIncrement();
}
/* Store dependencies on anything mentioned in predicate */
if (indexInfo->ii_Predicate)
{
+ /* recordDependencyOnSingleRelExpr get rid of duplicate entries */
recordDependencyOnSingleRelExpr(&myself,
(Node *) indexInfo->ii_Predicate,
heapRelationId,
DEPENDENCY_NORMAL,
- DEPENDENCY_AUTO, false);
+ DEPENDENCY_AUTO, false, true);
}
}
else
@@ -1229,6 +1262,93 @@ index_create(Relation heapRelation,
return indexRelationId;
}
+static char *
+index_check_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ Oid relid = *(Oid *) userdata;
+ char *current_version;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ /* Compare with the current version. */
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+
+ /* XXX should we warn about "disappearing" versions? */
+ if (current_version)
+ {
+ /*
+ * We now support versioning for the underlying collation library on
+ * this system, or previous version is unknown.
+ */
+ if (!version || strcmp(version, "") == 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" with an unknown version, and the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ else if (strcmp(current_version, version) != 0)
+ {
+ ereport(WARNING,
+ (errmsg("index \"%s\" depends on collation \"%s\" version \"%s\", but the current version is \"%s\"",
+ get_rel_name(relid),
+ get_collation_name(otherObject->objectId),
+ version,
+ current_version),
+ errdetail("The index may be corrupted due to changes in sort order."),
+ errhint("REINDEX to avoid the risk of corruption.")));
+ }
+ }
+
+ return NULL;
+}
+
+void
+index_check_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_check_collation_version, &relid);
+}
+
+static char *
+index_update_collation_version(const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata)
+{
+ char *current_version = (char *) userdata;
+
+ /* We only care about dependencies on collations. */
+ if (otherObject->classId != CollationRelationId)
+ return NULL;
+
+ current_version = get_collation_version_for_oid(otherObject->objectId);
+ return current_version;
+}
+
+static void
+index_update_collation_versions(Oid relid)
+{
+ ObjectAddress object;
+ NameData current_version;
+
+ object.classId = RelationRelationId;
+ object.objectId = relid;
+ object.objectSubId = 0;
+ visitDependentObjects(&object, &index_update_collation_version,
+ ¤t_version);
+}
+
/*
* index_concurrently_create_copy
*
@@ -3605,6 +3725,9 @@ reindex_index(Oid indexId, bool skip_constraint_checks, char persistence,
/* Close rels, but keep locks */
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
+
+ /* Record the current versions of all depended-on collations. */
+ index_update_collation_versions(indexId);
}
/*
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index 3d2b1cc911..0eaefbd032 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -360,7 +360,7 @@ CreateConstraintEntry(const char *constraintName,
*/
recordDependencyOnSingleRelExpr(&conobject, conExpr, relId,
DEPENDENCY_NORMAL,
- DEPENDENCY_NORMAL, false);
+ DEPENDENCY_NORMAL, false, true);
}
/* Post creation hook for new constraint */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 7fdbdf0ae8..709e7f5530 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -19,16 +19,21 @@
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/pg_locale.h"
#include "utils/rel.h"
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced);
static bool isObjectPinned(const ObjectAddress *object, Relation rel);
@@ -44,22 +49,29 @@ recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior)
{
- recordMultipleDependencies(depender, referenced, NULL, 1, behavior);
+ recordMultipleDependencies(depender, referenced, 1, behavior, false);
}
/*
- * As recordDependencyOn(), but also capture a version string so that changes
- * in the referenced object can be detected. The meaning of the version
- * string depends on the referenced object. Currently it is used for
- * detecting changes in collation versions.
+ * Given a list of collations, record a dependency on its underlying collation
+ * version.
*/
-void
-recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior)
+void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations)
{
- recordMultipleDependencies(depender, referenced, version, 1, behavior);
+ ListCell *lc;
+
+ foreach(lc, collations)
+ {
+ ObjectAddress referenced;
+
+ referenced.classId = CollationRelationId;
+ referenced.objectId = lfirst_oid(lc);
+ referenced.objectSubId = 0;
+
+ recordMultipleDependencies(myself, &referenced, 1,
+ DEPENDENCY_NORMAL, true);
+ }
}
/*
@@ -69,9 +81,9 @@ recordDependencyOnVersion(const ObjectAddress *depender,
void
recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior)
+ DependencyType behavior,
+ bool track_version)
{
Relation dependDesc;
CatalogIndexState indstate;
@@ -79,6 +91,7 @@ recordMultipleDependencies(const ObjectAddress *depender,
int i;
bool nulls[Natts_pg_depend];
Datum values[Natts_pg_depend];
+ char *version = NULL;
if (nreferenced <= 0)
return; /* nothing to do */
@@ -97,12 +110,46 @@ recordMultipleDependencies(const ObjectAddress *depender,
for (i = 0; i < nreferenced; i++, referenced++)
{
+ bool ignore_systempin = false;
+
+ if (track_version)
+ {
+ /* Only dependency on a collation needs to be tracked */
+ if (referenced->classId == CollationRelationId)
+ {
+ /* C and POSIX collations don't require tracking the version */
+ if (referenced->objectId == C_COLLATION_OID ||
+ referenced->objectId == POSIX_COLLATION_OID)
+ continue;
+
+ /*
+ * We don't want to record redundant depedencies that are used
+ * to track versions to avoid redundant warnings in case of
+ * non-matching versions when those are checked. Note that
+ * callers have to take care of removing duplicated entries and
+ * calling CommandCounterIncrement() if the dependencies are
+ * registered in multiple calls.
+ */
+ if (dependencyExists(depender, referenced))
+ continue;
+
+ /*
+ * Default collation is pinned, so we need to force recording
+ * the dependency to store the version
+ */
+ if (referenced->objectId == DEFAULT_COLLATION_OID)
+ ignore_systempin = true;;
+ version = get_collation_version_for_oid(referenced->objectId);
+ }
+ }
+
/*
* If the referenced object is pinned by the system, there's no real
- * need to record dependencies on it. This saves lots of space in
- * pg_depend, so it's worth the time taken to check.
+ * need to record dependencies on it, unless we need to record a
+ * version. This saves lots of space in pg_depend, so it's worth the
+ * time taken to check.
*/
- if (!isObjectPinned(referenced, dependDesc))
+ if (ignore_systempin || !isObjectPinned(referenced, dependDesc))
{
/*
* Record the Dependency. Note we don't bother to check for
@@ -559,6 +606,54 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
return count;
}
+/* dependencyExists()
+ *
+ * Test if a record exists for the given depender and referenceds addresses.
+ */
+static bool dependencyExists(const ObjectAddress *depender,
+ const ObjectAddress *referenced)
+{
+ Relation depRel;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+ HeapTuple tup;
+ bool ret = false;
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->classId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(depender->objectId));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum(depender->objectSubId));
+
+ depRel = table_open(DependRelationId, RowExclusiveLock);
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (foundDep->refclassid == referenced->classId &&
+ foundDep->refobjid == referenced->objectId &&
+ foundDep->refobjsubid == referenced->objectSubId)
+ {
+ ret = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+ table_close(depRel, RowExclusiveLock);
+
+ return ret;
+}
+
/*
* isObjectPinned()
*
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 8d7572da51..6e91518694 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/relation.h"
#include "access/table.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -511,6 +512,74 @@ TypeCreate(Oid newTypeOid,
return address;
}
+/* Get a list of all distinct collations oid that the given type depends on. */
+List *
+GetTypeCollations(Oid typeoid, bool non_deterministic_only)
+{
+ List *result = NIL;
+ HeapTuple tuple;
+ Form_pg_type typeTup;
+
+ tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typeoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for type %u", typeoid);
+ typeTup = (Form_pg_type) GETSTRUCT(tuple);
+
+ if (OidIsValid(typeTup->typcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(typeTup->typcollation))
+ result = list_append_unique_oid(result, typeTup->typcollation);
+ }
+ else if (typeTup->typtype == TYPTYPE_COMPOSITE)
+ {
+ Relation rel = relation_open(typeTup->typrelid, AccessShareLock);
+ TupleDesc desc = RelationGetDescr(rel);
+
+ for (int i = 0; i < RelationGetNumberOfAttributes(rel); i++)
+ {
+ Form_pg_attribute att = TupleDescAttr(desc, i);
+
+ if (OidIsValid(att->attcollation))
+ {
+ if (!non_deterministic_only ||
+ !get_collation_isdeterministic(att->attcollation))
+ result = list_append_unique_oid(result, att->attcollation);
+ }
+ else
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(att->atttypid,
+ non_deterministic_only));
+ }
+
+ relation_close(rel, NoLock);
+ }
+ else if (typeTup->typtype == TYPTYPE_DOMAIN)
+ {
+ Assert(OidIsValid(typeTup->typbasetype));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typbasetype,
+ non_deterministic_only));
+ }
+ else if (typeTup->typtype == TYPTYPE_RANGE)
+ {
+ Oid rangeid = get_range_subtype(typeTup->oid);
+
+ Assert(OidIsValid(rangeid));
+
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(rangeid, non_deterministic_only));
+ }
+ else if (OidIsValid(typeTup->typelem))
+ result = list_concat_unique_oid(result,
+ GetTypeCollations(typeTup->typelem, non_deterministic_only));
+
+ ReleaseSysCache(tuple);
+
+ return result;
+}
+
/*
* GenerateTypeDependencies: build the dependencies needed for a type
*
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index d625d17bf4..2b7de111cd 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -31,6 +31,8 @@
#include "access/tableam.h"
#include "access/transam.h"
#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/namespace.h"
#include "catalog/pg_database.h"
#include "catalog/pg_inherits.h"
@@ -634,6 +636,35 @@ vacuum_open_relation(Oid relid, RangeVar *relation, int options,
rel_lock = false;
}
+ /*
+ * Perform version sanity checks on the relation underlying indexes if
+ * it's not a VACUUM FULL
+ */
+ if (!(options & VACOPT_FULL) && onerel && !IsSystemRelation(onerel) &&
+ onerel->rd_rel->relhasindex)
+ {
+ List *indexoidlist;
+ ListCell *l;
+
+ indexoidlist = RelationGetIndexList(onerel);
+ foreach(l, indexoidlist)
+ {
+ Oid indexoid = lfirst_oid(l);
+ Relation indexRelation;
+
+ indexRelation = index_open(indexoid, AccessShareLock);
+
+ /* Warn if any dependent collations' versions have moved. */
+ if (!indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
+ index_close(indexRelation, NoLock);
+ }
+ }
+
/* if relation is opened, leave */
if (onerel)
return onerel;
diff --git a/src/backend/optimizer/util/plancat.c b/src/backend/optimizer/util/plancat.c
index d82fc5ab8b..f2fc427fc8 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -28,6 +28,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
+#include "catalog/index.h"
#include "catalog/pg_am.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_statistic_ext.h"
@@ -198,6 +199,14 @@ get_relation_info(PlannerInfo *root, Oid relationObjectId, bool inhparent,
indexRelation = index_open(indexoid, lmode);
index = indexRelation->rd_index;
+ /* Warn if any dependent collations' versions have moved. */
+ if (!IsSystemRelation(relation) &&
+ !indexRelation->rd_version_checked)
+ {
+ index_check_collation_versions(indexoid);
+ indexRelation->rd_version_checked = true;
+ }
+
/*
* Ignore invalid indexes, since they can't safely be used for
* queries. Note that this is OK because the data structure we
diff --git a/src/backend/utils/adt/pg_locale.c b/src/backend/utils/adt/pg_locale.c
index 597c1241f9..123bfa6c01 100644
--- a/src/backend/utils/adt/pg_locale.c
+++ b/src/backend/utils/adt/pg_locale.c
@@ -57,7 +57,9 @@
#include "access/htup_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_control.h"
+#include "catalog/pg_database.h"
#include "mb/pg_wchar.h"
+#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/formatting.h"
#include "utils/hsearch.h"
@@ -1501,6 +1503,42 @@ get_collation_actual_version(char collprovider, const char *collcollate)
return collversion;
}
+/*
+ * Get provider-specific collation version string given a collation OID.
+ */
+char *
+get_collation_version_for_oid(Oid oid)
+{
+ HeapTuple tp;
+ char *version;
+
+ if (oid == DEFAULT_COLLATION_OID)
+ {
+ Form_pg_database dbform;
+
+ tp = SearchSysCache1(DATABASEOID, ObjectIdGetDatum(MyDatabaseId));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for database %u", MyDatabaseId);
+ dbform = (Form_pg_database) GETSTRUCT(tp);
+ version = get_collation_actual_version(COLLPROVIDER_LIBC,
+ NameStr(dbform->datcollate));
+ }
+ else
+ {
+ Form_pg_collation collform;
+
+ tp = SearchSysCache1(COLLOID, ObjectIdGetDatum(oid));
+ if (!HeapTupleIsValid(tp))
+ elog(ERROR, "cache lookup failed for collation %u", oid);
+ collform = (Form_pg_collation) GETSTRUCT(tp);
+ version = get_collation_actual_version(collform->collprovider,
+ NameStr(collform->collcollate));
+ }
+
+ ReleaseSysCache(tp);
+
+ return version;
+}
#ifdef USE_ICU
/*
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index ff70326474..bdf50ffe89 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -41,6 +41,7 @@
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
+#include "catalog/index.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
#include "catalog/pg_am.h"
@@ -5623,6 +5624,7 @@ load_relcache_init_file(bool shared)
rel->rd_idattr = NULL;
rel->rd_pubactions = NULL;
rel->rd_statvalid = false;
+ rel->rd_version_checked = false;
rel->rd_statlist = NIL;
rel->rd_fkeyvalid = false;
rel->rd_fkeylist = NIL;
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 77cf0612ed..6de17e25e8 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -156,7 +156,8 @@ extern void recordDependencyOnSingleRelExpr(const ObjectAddress *depender,
Node *expr, Oid relId,
DependencyType behavior,
DependencyType self_behavior,
- bool reverse_self);
+ bool reverse_self,
+ bool track_version);
extern ObjectClass getObjectClass(const ObjectAddress *object);
@@ -176,22 +177,28 @@ extern void sort_object_addresses(ObjectAddresses *addrs);
extern void free_object_addresses(ObjectAddresses *addrs);
+typedef char *(*VisitDependentObjectsFun) (const ObjectAddress *otherObject,
+ const char *version,
+ void *userdata);
+
+extern void visitDependentObjects(const ObjectAddress *object,
+ VisitDependentObjectsFun callback,
+ void *userdata);
+
/* in pg_depend.c */
extern void recordDependencyOn(const ObjectAddress *depender,
const ObjectAddress *referenced,
DependencyType behavior);
-extern void recordDependencyOnVersion(const ObjectAddress *depender,
- const ObjectAddress *referenced,
- const NameData *version,
- DependencyType behavior);
+extern void recordDependencyOnCollations(ObjectAddress *myself,
+ List *collations);
extern void recordMultipleDependencies(const ObjectAddress *depender,
const ObjectAddress *referenced,
- const NameData *version,
int nreferenced,
- DependencyType behavior);
+ DependencyType behavior,
+ bool track_version);
extern void recordDependencyOnCurrentExtension(const ObjectAddress *object,
bool isReplace);
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index a2890c1314..c619d02465 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -121,6 +121,8 @@ extern void FormIndexDatum(IndexInfo *indexInfo,
Datum *values,
bool *isnull);
+extern void index_check_collation_versions(Oid relid);
+
extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index e1a5ab3df3..2bf6f868a5 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -335,6 +335,8 @@ extern void GenerateTypeDependencies(Oid typeObjectId,
bool isDependentType,
bool rebuild);
+extern List *GetTypeCollations(Oid typeObjectid, bool non_deterministic_only);
+
extern void RenameTypeInternal(Oid typeOid, const char *newTypeName,
Oid typeNamespace);
diff --git a/src/include/utils/pg_locale.h b/src/include/utils/pg_locale.h
index 9cb7d91ddf..aa06aca90a 100644
--- a/src/include/utils/pg_locale.h
+++ b/src/include/utils/pg_locale.h
@@ -104,6 +104,7 @@ typedef struct pg_locale_struct *pg_locale_t;
extern pg_locale_t pg_newlocale_from_collation(Oid collid);
extern char *get_collation_actual_version(char collprovider, const char *collcollate);
+extern char *get_collation_version_for_oid(Oid collid);
#ifdef USE_ICU
extern int32_t icu_to_uchar(UChar **buff_uchar, const char *buff, size_t nbytes);
diff --git a/src/include/utils/rel.h b/src/include/utils/rel.h
index 44ed04dd3f..3656ea94e8 100644
--- a/src/include/utils/rel.h
+++ b/src/include/utils/rel.h
@@ -63,6 +63,7 @@ typedef struct RelationData
bool rd_indexvalid; /* is rd_indexlist valid? (also rd_pkindex and
* rd_replidindex) */
bool rd_statvalid; /* is rd_statlist valid? */
+ bool rd_version_checked; /* has version check being done yet? */
/*
* rd_createSubid is the ID of the highest subtransaction the rel has
diff --git a/src/test/Makefile b/src/test/Makefile
index efb206aa75..4ef0d11702 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -12,7 +12,8 @@ subdir = src/test
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = perl regress isolation modules authentication recovery subscription
+SUBDIRS = perl regress isolation modules authentication recovery subscription \
+ locale
# Test suites that are not safe by default but can be run if selected
# by the user via the whitespace-separated list in variable
@@ -37,7 +38,7 @@ endif
# clean" etc to recurse into them. (We must filter out those that we
# have conditionally included into SUBDIRS above, else there will be
# make confusion.)
-ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap locale thread ssl)
+ALWAYS_SUBDIRS = $(filter-out $(SUBDIRS),examples kerberos ldap thread ssl)
# We want to recurse to all subdirs for all standard targets, except that
# installcheck and install should not recurse into the subdirectory "modules".
diff --git a/src/test/locale/.gitignore b/src/test/locale/.gitignore
index 620d3df425..64e1bf2a80 100644
--- a/src/test/locale/.gitignore
+++ b/src/test/locale/.gitignore
@@ -1 +1,2 @@
/test-ctype
+/tmp_check/
diff --git a/src/test/locale/Makefile b/src/test/locale/Makefile
index 22a45b65f2..73495cf16b 100644
--- a/src/test/locale/Makefile
+++ b/src/test/locale/Makefile
@@ -4,6 +4,7 @@ subdir = src/test/locale
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
+export with_icu
PROGS = test-ctype
DIRS = de_DE.ISO8859-1 gr_GR.ISO8859-7 koi8-r koi8-to-win1251
@@ -19,3 +20,9 @@ clean distclean maintainer-clean:
# These behave like installcheck targets.
check-%: all
@$(MAKE) -C `echo $@ | sed 's/^check-//'` test
+
+check:
+ $(prove_check)
+
+installcheck:
+ $(prove_installcheck)
diff --git a/src/test/locale/t/001_index.pl b/src/test/locale/t/001_index.pl
new file mode 100644
index 0000000000..468fbb63b6
--- /dev/null
+++ b/src/test/locale/t/001_index.pl
@@ -0,0 +1,76 @@
+use strict;
+use warnings;
+
+use Config;
+use PostgresNode;
+use TestLib;
+use Test::More;
+
+if ($ENV{with_icu} eq 'yes')
+{
+ plan tests => 12;
+}
+else
+{
+ plan skip_all => 'ICU not supported by this build';
+}
+
+#### Set up the server
+
+note "setting up data directory";
+my $node = get_new_node('main');
+$node->init;
+
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+sub test_index
+{
+ my ($err_like, $err_comm) = @_;
+
+ my ($ret, $out, $err) = $node->psql('postgres',
+ 'SET enable_seqscan = 0;'
+ . "EXPLAIN SELECT val FROM icu1 WHERE val = '0'");
+
+ is($ret, 0, 'EXPLAIN should succeed.');
+ like($out, qr/icu1_fr/, 'Index icu1_fr should be used.');
+ like($err, $err_like, $err_comm);
+}
+
+$node->safe_psql('postgres',
+ 'CREATE TABLE icu1(val text);'
+ . 'INSERT INTO icu1 SELECT i::text FROM generate_series(1, 10000) i;'
+ . 'CREATE INDEX icu1_fr ON icu1 (val COLLATE "fr-x-icu");');
+$node->safe_psql('postgres', 'VACUUM ANALYZE icu1;');
+
+test_index(qr/^$/, 'No warning should be raised');
+
+# Simulate different collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = 'not_a_version'"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" version "not_a_version", but the current version is/,
+ 'Different collation version warning should be raised.');
+
+# Simulate unknown collation version
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = ''"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+# Simulate previously unhandled collation versioning
+$node->safe_psql('postgres',
+ "UPDATE pg_depend SET refobjversion = NULL"
+ . " WHERE refobjversion IS NOT NULL"
+ . " AND objid::regclass::text = 'icu1_fr';");
+
+test_index(qr/index "icu1_fr" depends on collation "fr-x-icu" with an unknown version, and the current version is/,
+ 'Unknown collation version warning should be raised.');
+
+$node->stop;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out
index 60d9263a2f..f049189897 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8.out
@@ -1897,6 +1897,124 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- version of default (and libc) collation is only tracked on glibc systems
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'no version'
+WHEN refobjversion = '' THEN 'unknown version'
+ELSE 'some version' END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+--------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | some version
+ icuidx01_t_en_fr__d_es | "es-x-icu" | some version
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | some version
+ icuidx02_d_en_fr | "en-x-icu" | some version
+ icuidx02_d_en_fr | "fr-x-icu" | some version
+ icuidx03_t_en_fr_ga | "en-x-icu" | some version
+ icuidx03_t_en_fr_ga | "fr-x-icu" | some version
+ icuidx03_t_en_fr_ga | "ga-x-icu" | some version
+ icuidx04_d_en_fr_ga | "en-x-icu" | some version
+ icuidx04_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx04_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | some version
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | some version
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | some version
+ icuidx06_d_en_fr_ga | "default" | no version
+ icuidx06_d_en_fr_ga | "en-x-icu" | some version
+ icuidx06_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx06_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx07_d_en_fr_ga | "default" | no version
+ icuidx07_d_en_fr_ga | "en-x-icu" | some version
+ icuidx07_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx07_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx08_d_en_fr_ga | "en-x-icu" | some version
+ icuidx08_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx08_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx09_d_en_fr_ga | "en-x-icu" | some version
+ icuidx09_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx09_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | some version
+ icuidx11_d_es | "default" | no version
+ icuidx11_d_es | "es-x-icu" | some version
+ icuidx12_custom | "default" | no version
+ icuidx12_custom | custom | some version
+ icuidx13_custom | "default" | no version
+ icuidx13_custom | custom | some version
+ icuidx14_myrange | "default" | no version
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | some version
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | some version
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | some version
+ icuidx16_mood | "fr-x-icu" | some version
+ icuidx17_part | "en-x-icu" | some version
+(44 rows)
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8_2.out
similarity index 89%
copy from src/test/regress/expected/collate.icu.utf8.out
copy to src/test/regress/expected/collate.icu.utf8_2.out
index 60d9263a2f..07e76c4f48 100644
--- a/src/test/regress/expected/collate.icu.utf8.out
+++ b/src/test/regress/expected/collate.icu.utf8_2.out
@@ -1897,6 +1897,124 @@ SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
t
(1 row)
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+-- version of default (and libc) collation is only tracked on glibc systems
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'no version'
+WHEN refobjversion = '' THEN 'unknown version'
+ELSE 'some version' END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+ objid | refobjid | version
+---------------------------+------------+--------------
+ icuidx01_t_en_fr__d_es | "en-x-icu" | some version
+ icuidx01_t_en_fr__d_es | "es-x-icu" | some version
+ icuidx01_t_en_fr__d_es | "fr-x-icu" | some version
+ icuidx02_d_en_fr | "en-x-icu" | some version
+ icuidx02_d_en_fr | "fr-x-icu" | some version
+ icuidx03_t_en_fr_ga | "en-x-icu" | some version
+ icuidx03_t_en_fr_ga | "fr-x-icu" | some version
+ icuidx03_t_en_fr_ga | "ga-x-icu" | some version
+ icuidx04_d_en_fr_ga | "en-x-icu" | some version
+ icuidx04_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx04_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx05_d_en_fr_ga_arr | "en-x-icu" | some version
+ icuidx05_d_en_fr_ga_arr | "fr-x-icu" | some version
+ icuidx05_d_en_fr_ga_arr | "ga-x-icu" | some version
+ icuidx06_d_en_fr_ga | "default" | some version
+ icuidx06_d_en_fr_ga | "en-x-icu" | some version
+ icuidx06_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx06_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx07_d_en_fr_ga | "default" | some version
+ icuidx07_d_en_fr_ga | "en-x-icu" | some version
+ icuidx07_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx07_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx08_d_en_fr_ga | "en-x-icu" | some version
+ icuidx08_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx08_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx09_d_en_fr_ga | "en-x-icu" | some version
+ icuidx09_d_en_fr_ga | "fr-x-icu" | some version
+ icuidx09_d_en_fr_ga | "ga-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "en-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "es-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "fr-x-icu" | some version
+ icuidx10_d_en_fr_ga_es | "ga-x-icu" | some version
+ icuidx11_d_es | "default" | some version
+ icuidx11_d_es | "es-x-icu" | some version
+ icuidx12_custom | "default" | some version
+ icuidx12_custom | custom | some version
+ icuidx13_custom | "default" | some version
+ icuidx13_custom | custom | some version
+ icuidx14_myrange | "default" | some version
+ icuidx15_myrange_en_fr_ga | "en-x-icu" | some version
+ icuidx15_myrange_en_fr_ga | "fr-x-icu" | some version
+ icuidx15_myrange_en_fr_ga | "ga-x-icu" | some version
+ icuidx16_mood | "fr-x-icu" | some version
+ icuidx17_part | "en-x-icu" | some version
+(44 rows)
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
+ objid
+-------
+(0 rows)
+
-- cleanup
RESET search_path;
SET client_min_messages TO warning;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 6ddf3a63c3..92dcda5bdc 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2065,15 +2065,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,15 +2095,17 @@ WHERE classid = 'pg_class'::regclass AND
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql
index 35acf91fbf..920a73512f 100644
--- a/src/test/regress/sql/collate.icu.utf8.sql
+++ b/src/test/regress/sql/collate.icu.utf8.sql
@@ -716,6 +716,82 @@ INSERT INTO test33 VALUES (2, 'DEF');
-- they end up in the same partition (but it's platform-dependent which one)
SELECT (SELECT count(*) FROM test33_0) <> (SELECT count(*) FROM test33_1);
+-- collation versioning support
+CREATE TYPE t_en_fr AS (fr text COLLATE "fr-x-icu", en text COLLATE "en-x-icu");
+CREATE DOMAIN d_en_fr AS t_en_fr;
+CREATE DOMAIN d_es AS text COLLATE "es-x-icu";
+CREATE TYPE t_en_fr_ga AS (en_fr t_en_fr, ga text COLLATE "ga-x-icu");
+CREATE DOMAIN d_en_fr_ga AS t_en_fr_ga;
+CREATE TYPE t_custom AS (meh text, meh2 text);
+CREATE DOMAIN d_custom AS t_custom;
+
+CREATE COLLATION custom (
+ LOCALE = 'fr-x-icu', PROVIDER = 'icu'
+);
+
+CREATE TYPE myrange AS range (subtype = text);
+CREATE TYPE myrange_en_fr_ga AS range(subtype = t_en_fr_ga);
+
+CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
+
+CREATE TABLE collate_test (
+ id integer,
+ t_en_fr t_en_fr,
+ d_en_fr d_en_fr,
+ d_es d_es,
+ t_en_fr_ga t_en_fr_ga,
+ d_en_fr_ga d_en_fr_ga,
+ d_en_fr_ga_arr d_en_fr_ga[],
+ myrange myrange,
+ myrange_en_fr_ga myrange_en_fr_ga,
+ mood mood
+);
+
+CREATE INDEX icuidx01_t_en_fr__d_es ON collate_test (t_en_fr, d_es);
+CREATE INDEX icuidx02_d_en_fr ON collate_test (d_en_fr);
+CREATE INDEX icuidx03_t_en_fr_ga ON collate_test (t_en_fr_ga);
+CREATE INDEX icuidx04_d_en_fr_ga ON collate_test (d_en_fr_ga);
+CREATE INDEX icuidx05_d_en_fr_ga_arr ON collate_test (d_en_fr_ga_arr);
+CREATE INDEX icuidx06_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).en_fr.fr = 'foo';
+CREATE INDEX icuidx07_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga).ga = 'foo';
+CREATE INDEX icuidx08_d_en_fr_ga ON collate_test(id) WHERE (t_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx09_d_en_fr_ga ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz');
+CREATE INDEX icuidx10_d_en_fr_ga_es ON collate_test(id) WHERE (d_en_fr_ga) = ('foo', 'bar', 'baz' COLLATE "es-x-icu");
+CREATE INDEX icuidx11_d_es ON collate_test(id) WHERE (d_es) = ('foo');
+CREATE INDEX icuidx12_custom ON collate_test(id) WHERE ('foo', 'bar')::d_custom = ('foo', 'bar' collate custom)::d_custom;
+CREATE INDEX icuidx13_custom ON collate_test(id) WHERE ('foo' collate custom, 'bar')::d_custom = ('foo', 'bar')::d_custom;
+CREATE INDEX icuidx14_myrange ON collate_test(myrange);
+CREATE INDEX icuidx15_myrange_en_fr_ga ON collate_test USING gist (myrange_en_fr_ga);
+CREATE INDEX icuidx16_mood ON collate_test(id) WHERE mood > 'ok' collate "fr-x-icu";
+
+CREATE TABLE collate_part(id integer, val text COLLATE "en-x-icu") PARTITION BY range(id);
+CREATE TABLE collate_part_0 PARTITION OF collate_part FOR VALUES FROM (0) TO (1);
+CREATE TABLE collate_part_1 PARTITION OF collate_part FOR VALUES FROM (1) TO (1000000);
+CREATE INDEX icuidx17_part ON collate_part_1 (val);
+
+-- version of default (and libc) collation is only tracked on glibc systems
+SELECT objid::regclass, refobjid::regcollation,
+CASE
+WHEN refobjversion IS NULL THEN 'no version'
+WHEN refobjversion = '' THEN 'unknown version'
+ELSE 'some version' END AS version
+FROM pg_depend d
+LEFT JOIN pg_class c ON c.oid = d.objid
+WHERE refclassid = 'pg_collation'::regclass
+AND coalesce(relkind, 'i') = 'i'
+AND relname LIKE 'icuidx%'
+ORDER BY 1, 2;
+
+UPDATE pg_depend SET refobjversion = 'not a version'
+WHERE refclassid = 'pg_collation'::regclass
+AND objid::regclass::text LIKE 'icuidx%'
+AND refobjversion IS NOT NULL;
+
+REINDEX TABLE collate_test;
+REINDEX TABLE collate_part_0;
+REINDEX TABLE collate_part_1;
+
+SELECT objid::regclass FROM pg_depend WHERE refobjversion = 'not a version';
-- cleanup
RESET search_path;
--
2.20.1
--Q68bSM7Ycu6FN28Q
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment;
filename*0="0005-Preserve-index-dependencies-on-collation-during-pg_u-v9.";
filename*1=patch
^ permalink raw reply [nested|flat] 98+ messages in thread
* RE: Idea to enhance pgbench by more modes to generate data (multi-TXNs, UNNEST, COPY BINARY)
@ 2026-01-30 13:22 Madyshev Egor <[email protected]>
2026-01-30 14:47 ` Re: Idea to enhance pgbench by more modes to generate data (multi-TXNs, UNNEST, COPY BINARY) Boris Mironov <[email protected]>
0 siblings, 1 reply; 98+ messages in thread
From: Madyshev Egor @ 2026-01-30 13:22 UTC (permalink / raw)
To: Boris Mironov <[email protected]>; pgsql-hackers
Hello Boris,
I've reviewed and tested your patch. In some modes, I did observe a
performance improvement. However, in my opinion, the current set of
modes is not transparent enough.
I propose, by analogy with the existing 'g'/'G' modes, to use lowercase
letters for client-side data generation and uppercase letters for
server-side generation. Furthermore, I propose considering making the
"one transaction per scale" mode a separate setting. This would result
in the following modes:
1. g: COPY .. FROM STDIN .. TEXT, single transaction (orig. mode)
2. c: COPY .. FROM STDIN .. BINARY, single transaction (added mode)
3. G: INSERT .. SELECT generate_series, single transaction (orig. mode)
4. I: INSERT .. SELECT unnest, single transaction (added mode)
And: M: multiple transactions. A setting that, when used, makes a mode
run with a transaction for each scale instead of a single transaction.
This would yield 8 possible combinations.
It would be reasonable to first collect performance measurements for
these modes and then decide whether to keep them, before proceeding with
a full implementation including their selection.
The provided measurements do not seem conclusive. For example, in mode
G(1) at scale 200, the result was 46.64, while the next measurement for
G(2) was 56.6, while mode i(3) falls between them at 47.63. Could you
please describe how you collected the performance measurements? Is it
possible that the measurement deviations significantly affected the
results shown in the table? It would be correct to take measurements
several times and then present a table with averages.
> I also switched from one huge transaction for COPY FROM BINARY to
> 'one per scale'. This will simplify merge with multi-threaded data load
> proposed by Mircea. Unfortunately, it killed possibility to freeze
> data right away, which was possible when table truncation and data load
> was done in the same transaction.
It seems incorrect to me to make a decision to abandon the freeze
optimization solely because of another patch, especially one that is
not yet in master. Please provide more motivation on why using one
transaction per scale is more beneficial than a single transaction
combined with the freeze optimization. Having a setting to switch
the transaction mode would allow avoiding this trade-off.
Thus, I propose reconsidering the approach to data generation modes
and adding a setting to control the number of transactions.
I also suggest conducting new, more accurate performance measurements to
inform the decision on the necessity of the additional generation modes.
Best regards,
Egor
^ permalink raw reply [nested|flat] 98+ messages in thread
* Re: Idea to enhance pgbench by more modes to generate data (multi-TXNs, UNNEST, COPY BINARY)
2026-01-30 13:22 RE: Idea to enhance pgbench by more modes to generate data (multi-TXNs, UNNEST, COPY BINARY) Madyshev Egor <[email protected]>
@ 2026-01-30 14:47 ` Boris Mironov <[email protected]>
2026-02-06 10:48 ` Re: Idea to enhance pgbench by more modes to generate data (multi-TXNs, UNNEST, COPY BINARY) Boris Mironov <[email protected]>
0 siblings, 1 reply; 98+ messages in thread
From: Boris Mironov @ 2026-01-30 14:47 UTC (permalink / raw)
To: Madyshev Egor <[email protected]>; pgsql-hackers
Hello Egor,
Thank you very much for taking this patch under your wing!
> I propose, by analogy with the existing 'g'/'G' modes, to use lowercase
> letters for client-side data generation and uppercase letters for
> server-side generation. Furthermore, I propose considering making the
> "one transaction per scale" mode a separate setting. This would result
> in the following modes:
> 1. g: COPY .. FROM STDIN .. TEXT, single transaction (orig. mode)
> 2. c: COPY .. FROM STDIN .. BINARY, single transaction (added mode)
> 3. G: INSERT .. SELECT generate_series, single transaction (orig. mode)
> 4. I: INSERT .. SELECT unnest, single transaction (added mode)
> And: M: multiple transactions. A setting that, when used, makes a mode
> run with a transaction for each scale instead of a single transaction.
> This would yield 8 possible combinations.
Sure thing. I agree with your proposal to add more flexibility to
parameters with single exception. For UNNEST test I would suggest
to use "U" instead of "I" as it might be confusing later in case of
another patch from current CommitFest will make it into the master.
I'm referring to:
https://commitfest.postgresql.org/patch/6242/
It uses parameter "-i" to start use multiple threads to populate tables.
> It would be reasonable to first collect performance measurements for
> these modes and then decide whether to keep them, before proceeding with
> a full implementation including their selection.
Since logic will be slightly different by following your proposal new set
of metrics will be required. That's for sure.
My main motivation in splitting one huge transaction to fill tables
into smaller ones comes from another idea that was put on
a backburner - running data population via multiple threads. This
idea is implemented in above mentioned patch by Mircea Cadariu.
By amount of changes in that patch it is clear that we're quite equal
by number of lines. Hence putting the change into my patch would
be overwhelming for any reviewer.
Another reason for smaller in size transactions ("one per scale")
is my experience during generation of test databases that are much
bigger than host RAM (e.g., scale=5000). Data population phase is
not just slow, but more than often has to use multiple checkpoints
for such single transaction because even my max_wal_size was
smaller than size of such "change". One might argue that my DB
is not tuned properly, but it's a topic for another day. As a side
effect of decision to use multiple transactions raises another
issue - inability to use FREEZE optimisation for COPY commands
which leads to Autovacuum storm in turn even during very process
of data population.
> Thus, I propose reconsidering the approach to data generation modes
> and adding a setting to control the number of transactions.
> I also suggest conducting new, more accurate performance measurements to
> inform the decision on the necessity of the additional generation modes.
Agree and Agree. Both make perfect sense.
Best regards,
Boris
^ permalink raw reply [nested|flat] 98+ messages in thread
* Re: Idea to enhance pgbench by more modes to generate data (multi-TXNs, UNNEST, COPY BINARY)
2026-01-30 13:22 RE: Idea to enhance pgbench by more modes to generate data (multi-TXNs, UNNEST, COPY BINARY) Madyshev Egor <[email protected]>
2026-01-30 14:47 ` Re: Idea to enhance pgbench by more modes to generate data (multi-TXNs, UNNEST, COPY BINARY) Boris Mironov <[email protected]>
@ 2026-02-06 10:48 ` Boris Mironov <[email protected]>
2026-02-06 11:08 ` Re: Idea to enhance pgbench by more modes to generate data (multi-TXNs, UNNEST, COPY BINARY) Boris Mironov <[email protected]>
0 siblings, 1 reply; 98+ messages in thread
From: Boris Mironov @ 2026-02-06 10:48 UTC (permalink / raw)
To: Madyshev Egor <[email protected]>; pgsql-hackers
Hi Egor,
As per your recommendations I've updated code and attached it below.
I've also prepared Excel spreadsheet with benchmarks in raw format as well as processed for better presentation.
Just for reference I provide my test configuration and benchmarks.
Setup
VM: 4 CPU 2.2 GHz Intel Xeon Silver 4114, 16 GB RAM
shared_buffers=2GB
wal_keep_size=300MB
Same DB for all tests. No replicas, just primary instance
Scale | 1 | 2 | 10 | 20 | 100 | 200 | 1000 | 2000
DB size | 1761 MB | 1774 MB | 1878 MB | 2009 MB | 3050 MB | 4351 MB | 14 GB | 27 GB
Data generation modes
Prefixes S Single transaction for whole data set
M Multiple transactions (as 1 transaction per "scale")
Modes G INSERT .. FROM generate_series
U INSERT .. FROM unnest
g COPY .. FROM STDIN TEXT
c COPY .. FROM STDIN BINARY
Average time of 5 runs to complete data initialization
pgbench (PostgreSQL) 17.7 (Debian 17.7-3.pgdg12+1)         
Mode
Scale | 1 | 2 | 10 | 20 | 100 | 200 | 1000 | 2000
SG | 0.22 | 0.45 | 2.18 | 4.39 | 23.94 | 47.89 | 241.99 | 550.91
Sg | 0.16 | 0.27 | 1.53 | 3.19 | 16.08 | 32.98 | 161.31 | 327.56
pgbench (PostgreSQL) 19devel / client-side generation
Mode
Scale | 1 | 2 | 10 | 20 | 100 | 200 | 1000 | 2000
Mc | 0.15 | 0.29 | 1.39 | 2.93 | 14.79 | 30.78 | 161.52 | 330.63
Sc | 0.13 | 0.27 | 1.37 | 2.69 | 14.71 | 29.99 | 152.83 | 298.88
Mg | 0.17 | 0.30 | 1.58 | 3.26 | 15.91 | 31.95 | 160.31 | 326.46
Sg | 0.20 | 0.38 | 1.66 | 3.39 | 18.72 | 36.26 | 176.54 | 351.66
pgbench (PostgreSQL) 19devel / server-side generation
Mode
Scale | 1 | 2 | 10 | 20 | 100 | 200 | 1000 | 2000
MU | 0.22 | 0.47 | 2.35 | 4.71 | 24.80 | 53.19 | 261.44 | 536.05
SU | 0.22 | 0.44 | 2.35 | 4.78 | 24.42 | 49.16 | 246.41 | 495.51
MG | 0.22 | 0.45 | 2.29 | 4.44 | 24.03 | 50.21 | 256.54 | 544.17
SG | 0.23 | 0.43 | 2.35 | 5.04 | 27.74 | 52.72 | 250.27 | 492.77
Best regards,
Boris
Attachments:
[application/vnd.openxmlformats-officedocument.spreadsheetml.sheet] performance_tests.xlsx (38.8K, ../../PH0PR08MB70204888D724794BA83EA6E58866A@PH0PR08MB7020.namprd08.prod.outlook.com/3-performance_tests.xlsx)
download
[application/octet-stream] v7-pgbench-faster-modes.patch (81.3K, ../../PH0PR08MB70204888D724794BA83EA6E58866A@PH0PR08MB7020.namprd08.prod.outlook.com/4-v7-pgbench-faster-modes.patch)
download | inline diff:
From c768f399c556295de7d53895410e686d86b4b960 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Sun, 9 Nov 2025 19:34:58 +0700
Subject: [PATCH v7 01/18] Converting one huge transaction into series of one
per 'scale'
---
src/bin/pgbench/pgbench.c | 61 ++++++++++++++++++++++++++-------------
1 file changed, 41 insertions(+), 20 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index d8764ba6fe0..284a7c860f1 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -181,6 +181,12 @@ static int64 end_time = 0; /* when to stop in micro seconds, under -T */
*/
static int scale = 1;
+/*
+ * scaling factor after which we switch to multiple transactions during
+ * data population phase on server side
+ */
+static int64 single_txn_scale_limit = 1;
+
/*
* fillfactor. for example, fillfactor = 90 will use only 90 percent
* space during inserts and leave 10 percent free.
@@ -5213,6 +5219,7 @@ static void
initGenerateDataServerSide(PGconn *con)
{
PQExpBufferData sql;
+ int chunk = (scale >= single_txn_scale_limit) ? 1 : scale;
fprintf(stderr, "generating data (server-side)...\n");
@@ -5225,30 +5232,44 @@ initGenerateDataServerSide(PGconn *con)
/* truncate away any old data */
initTruncateTables(con);
+ executeStatement(con, "commit");
+
initPQExpBuffer(&sql);
- printfPQExpBuffer(&sql,
- "insert into pgbench_branches(bid,bbalance) "
- "select bid, 0 "
- "from generate_series(1, %d) as bid", nbranches * scale);
- executeStatement(con, sql.data);
-
- printfPQExpBuffer(&sql,
- "insert into pgbench_tellers(tid,bid,tbalance) "
- "select tid, (tid - 1) / %d + 1, 0 "
- "from generate_series(1, %d) as tid", ntellers, ntellers * scale);
- executeStatement(con, sql.data);
-
- printfPQExpBuffer(&sql,
- "insert into pgbench_accounts(aid,bid,abalance,filler) "
- "select aid, (aid - 1) / %d + 1, 0, '' "
- "from generate_series(1, " INT64_FORMAT ") as aid",
- naccounts, (int64) naccounts * scale);
- executeStatement(con, sql.data);
+ for (int i = 0; i < scale; i += chunk) {
+ executeStatement(con, "begin");
+
+ printfPQExpBuffer(&sql,
+ "insert into pgbench_branches(bid,bbalance) "
+ "select bid + 1, 0 "
+ "from generate_series(%d, %d) as bid", i, i + chunk);
+ //"select bid, 0 "
+ //"from generate_series(1, %d) as bid", nbranches * scale);
+ executeStatement(con, sql.data);
+
+ printfPQExpBuffer(&sql,
+ "insert into pgbench_tellers(tid,bid,tbalance) "
+ "select tid + 1, tid / %d + 1, 0 "
+ "from generate_series(%d, %d) as tid",
+ ntellers, i * ntellers, (i + chunk) * ntellers - 1);
+ //"select tid, (tid - 1) / %d + 1, 0 "
+ //"from generate_series(1, %d) as tid", ntellers, ntellers * scale);
+ executeStatement(con, sql.data);
+
+ printfPQExpBuffer(&sql,
+ "insert into pgbench_accounts(aid,bid,abalance,filler) "
+ "select aid + 1, aid / %d + 1, 0, '' "
+ "from generate_series(" INT64_FORMAT ", " INT64_FORMAT ") as aid",
+ naccounts, (int64) i * naccounts, (int64) (i + chunk) * naccounts - 1);
+ //"select aid, (aid - 1) / %d + 1, 0, '' "
+ //"from generate_series(1, " INT64_FORMAT ") as aid",
+ //naccounts, (int64) naccounts * scale);
+ executeStatement(con, sql.data);
+
+ executeStatement(con, "commit");
+ }
termPQExpBuffer(&sql);
-
- executeStatement(con, "commit");
}
/*
--
2.43.0
From 0eddb156c187d829c4381bc928c5314705928852 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Sun, 9 Nov 2025 20:13:23 +0700
Subject: [PATCH v7 02/18] Getting rid off limit for single transaction size
during data generation
---
src/bin/pgbench/pgbench.c | 15 ++++-----------
1 file changed, 4 insertions(+), 11 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 284a7c860f1..28b72e4cf1f 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -181,12 +181,6 @@ static int64 end_time = 0; /* when to stop in micro seconds, under -T */
*/
static int scale = 1;
-/*
- * scaling factor after which we switch to multiple transactions during
- * data population phase on server side
- */
-static int64 single_txn_scale_limit = 1;
-
/*
* fillfactor. for example, fillfactor = 90 will use only 90 percent
* space during inserts and leave 10 percent free.
@@ -5219,7 +5213,6 @@ static void
initGenerateDataServerSide(PGconn *con)
{
PQExpBufferData sql;
- int chunk = (scale >= single_txn_scale_limit) ? 1 : scale;
fprintf(stderr, "generating data (server-side)...\n");
@@ -5236,13 +5229,13 @@ initGenerateDataServerSide(PGconn *con)
initPQExpBuffer(&sql);
- for (int i = 0; i < scale; i += chunk) {
+ for (int i = 0; i < scale; i++) {
executeStatement(con, "begin");
printfPQExpBuffer(&sql,
"insert into pgbench_branches(bid,bbalance) "
"select bid + 1, 0 "
- "from generate_series(%d, %d) as bid", i, i + chunk);
+ "from generate_series(%d, %d) as bid", i, i + 1);
//"select bid, 0 "
//"from generate_series(1, %d) as bid", nbranches * scale);
executeStatement(con, sql.data);
@@ -5251,7 +5244,7 @@ initGenerateDataServerSide(PGconn *con)
"insert into pgbench_tellers(tid,bid,tbalance) "
"select tid + 1, tid / %d + 1, 0 "
"from generate_series(%d, %d) as tid",
- ntellers, i * ntellers, (i + chunk) * ntellers - 1);
+ ntellers, i * ntellers, (i + 1) * ntellers - 1);
//"select tid, (tid - 1) / %d + 1, 0 "
//"from generate_series(1, %d) as tid", ntellers, ntellers * scale);
executeStatement(con, sql.data);
@@ -5260,7 +5253,7 @@ initGenerateDataServerSide(PGconn *con)
"insert into pgbench_accounts(aid,bid,abalance,filler) "
"select aid + 1, aid / %d + 1, 0, '' "
"from generate_series(" INT64_FORMAT ", " INT64_FORMAT ") as aid",
- naccounts, (int64) i * naccounts, (int64) (i + chunk) * naccounts - 1);
+ naccounts, (int64) i * naccounts, (int64) (i + 1) * naccounts - 1);
//"select aid, (aid - 1) / %d + 1, 0, '' "
//"from generate_series(1, " INT64_FORMAT ") as aid",
//naccounts, (int64) naccounts * scale);
--
2.43.0
From c5659cf474ec273c057668f30a4f435fd02f2da7 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Sun, 9 Nov 2025 20:38:36 +0700
Subject: [PATCH v7 03/18] No need to keep old code in comments
---
src/bin/pgbench/pgbench.c | 7 -------
1 file changed, 7 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 28b72e4cf1f..97895aa9edf 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -5236,8 +5236,6 @@ initGenerateDataServerSide(PGconn *con)
"insert into pgbench_branches(bid,bbalance) "
"select bid + 1, 0 "
"from generate_series(%d, %d) as bid", i, i + 1);
- //"select bid, 0 "
- //"from generate_series(1, %d) as bid", nbranches * scale);
executeStatement(con, sql.data);
printfPQExpBuffer(&sql,
@@ -5245,8 +5243,6 @@ initGenerateDataServerSide(PGconn *con)
"select tid + 1, tid / %d + 1, 0 "
"from generate_series(%d, %d) as tid",
ntellers, i * ntellers, (i + 1) * ntellers - 1);
- //"select tid, (tid - 1) / %d + 1, 0 "
- //"from generate_series(1, %d) as tid", ntellers, ntellers * scale);
executeStatement(con, sql.data);
printfPQExpBuffer(&sql,
@@ -5254,9 +5250,6 @@ initGenerateDataServerSide(PGconn *con)
"select aid + 1, aid / %d + 1, 0, '' "
"from generate_series(" INT64_FORMAT ", " INT64_FORMAT ") as aid",
naccounts, (int64) i * naccounts, (int64) (i + 1) * naccounts - 1);
- //"select aid, (aid - 1) / %d + 1, 0, '' "
- //"from generate_series(1, " INT64_FORMAT ") as aid",
- //naccounts, (int64) naccounts * scale);
executeStatement(con, sql.data);
executeStatement(con, "commit");
--
2.43.0
From e47b52ddf23593dad9375ef5356fd41d0621ede3 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Mon, 10 Nov 2025 19:06:48 +0700
Subject: [PATCH v7 04/18] Adding server-side data generation via unnest
---
src/bin/pgbench/pgbench.c | 199 ++++++++++++++++++++++++++++++++++----
1 file changed, 182 insertions(+), 17 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 97895aa9edf..65d77cdefea 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -161,7 +161,7 @@ typedef struct socket_set
* some configurable parameters */
#define DEFAULT_INIT_STEPS "dtgvp" /* default -I setting */
-#define ALL_INIT_STEPS "dtgGvpf" /* all possible steps */
+#define ALL_INIT_STEPS "dtgGiIvpf" /* all possible steps */
#define LOG_STEP_SECONDS 5 /* seconds between log messages */
#define DEFAULT_NXACTS 10 /* default nxacts */
@@ -171,6 +171,12 @@ typedef struct socket_set
#define MIN_ZIPFIAN_PARAM 1.001 /* minimum parameter for zipfian */
#define MAX_ZIPFIAN_PARAM 1000.0 /* maximum parameter for zipfian */
+/* original single transaction server-side method */
+#define GEN_TYPE_INSERT_ORIGINAL 'G' /* use INSERT .. SELECT generate_series to generate data */
+/* 'one transaction per scale' server-side methods */
+#define GEN_TYPE_INSERT_SERIES 'i' /* use INSERT .. SELECT generate_series to generate data */
+#define GEN_TYPE_INSERT_UNNEST 'I' /* use INSERT .. SELECT unnest to generate data */
+
static int nxacts = 0; /* number of transactions per client */
static int duration = 0; /* duration in seconds */
static int64 end_time = 0; /* when to stop in micro seconds, under -T */
@@ -181,6 +187,11 @@ static int64 end_time = 0; /* when to stop in micro seconds, under -T */
*/
static int scale = 1;
+/*
+ *
+ */
+static char data_generation_type = '?';
+
/*
* fillfactor. for example, fillfactor = 90 will use only 90 percent
* space during inserts and leave 10 percent free.
@@ -914,7 +925,9 @@ usage(void)
" d: drop any existing pgbench tables\n"
" t: create the tables used by the standard pgbench scenario\n"
" g: generate data, client-side\n"
- " G: generate data, server-side\n"
+ " G: generate data, server-side in single transaction\n"
+ " i: server-side (multiple TXNs) INSERT .. SELECT generate_series\n"
+ " I: server-side (multiple TXNs) INSERT .. SELECT unnest\n"
" v: invoke VACUUM on the standard tables\n"
" p: create primary key indexes on the standard tables\n"
" f: create foreign keys between the standard tables\n"
@@ -5203,18 +5216,16 @@ initGenerateDataClientSide(PGconn *con)
}
/*
- * Fill the standard tables with some data generated on the server
- *
- * As already the case with the client-side data generation, the filler
- * column defaults to NULL in pgbench_branches and pgbench_tellers,
- * and is a blank-padded string in pgbench_accounts.
+ * Generating data via INSERT .. SELECT .. FROM generate_series
+ * whole dataset in single transaction
*/
static void
-initGenerateDataServerSide(PGconn *con)
+generateDataInsertSingleTXN(PGconn *con)
{
PQExpBufferData sql;
- fprintf(stderr, "generating data (server-side)...\n");
+ fprintf(stderr, "via INSERT .. SELECT generate_series... in single TXN\n");
+
/*
* we do all of this in one transaction to enable the backend's
@@ -5225,31 +5236,136 @@ initGenerateDataServerSide(PGconn *con)
/* truncate away any old data */
initTruncateTables(con);
+ initPQExpBuffer(&sql);
+
+ printfPQExpBuffer(&sql,
+ "insert into pgbench_branches(bid, bbalance) "
+ "select bid, 0 "
+ "from generate_series(1, %d)", scale * nbranches);
+ executeStatement(con, sql.data);
+
+ printfPQExpBuffer(&sql,
+ "insert into pgbench_tellers(tid, bid, tbalance) "
+ "select tid + 1, tid / %d + 1, 0 "
+ "from generate_series(0, %d) as tid",
+ ntellers, (scale * ntellers) - 1);
+ executeStatement(con, sql.data);
+
+ printfPQExpBuffer(&sql,
+ "insert into pgbench_accounts(aid, bid, abalance, "
+ "filler) "
+ "select aid + 1, aid / %d + 1, 0, '' "
+ "from generate_series(0, " INT64_FORMAT ") as aid",
+ naccounts, (int64) (scale * naccounts) - 1);
+ executeStatement(con, sql.data);
+
executeStatement(con, "commit");
+ termPQExpBuffer(&sql);
+}
+
+
+/*
+ * Generating data via INSERT .. SELECT .. FROM generate_series
+ * One transaction per 'scale'
+ */
+static void
+generateDataInsertSeries(PGconn *con)
+{
+ PQExpBufferData sql;
+
+ fprintf(stderr, "via INSERT .. SELECT generate_series... in multiple TXN(s)\n");
+
initPQExpBuffer(&sql);
- for (int i = 0; i < scale; i++) {
+ executeStatement(con, "begin");
+
+ /* truncate away any old data */
+ initTruncateTables(con);
+
+ executeStatement(con, "commit");
+
+ for (int i = 0; i < scale; i++)
+ {
executeStatement(con, "begin");
printfPQExpBuffer(&sql,
- "insert into pgbench_branches(bid,bbalance) "
- "select bid + 1, 0 "
- "from generate_series(%d, %d) as bid", i, i + 1);
+ "insert into pgbench_branches(bid, bbalance) "
+ "values(%d, 0)", i + 1);
executeStatement(con, sql.data);
printfPQExpBuffer(&sql,
- "insert into pgbench_tellers(tid,bid,tbalance) "
+ "insert into pgbench_tellers(tid, bid, tbalance) "
"select tid + 1, tid / %d + 1, 0 "
"from generate_series(%d, %d) as tid",
ntellers, i * ntellers, (i + 1) * ntellers - 1);
executeStatement(con, sql.data);
printfPQExpBuffer(&sql,
- "insert into pgbench_accounts(aid,bid,abalance,filler) "
+ "insert into pgbench_accounts(aid, bid, abalance, "
+ "filler) "
"select aid + 1, aid / %d + 1, 0, '' "
- "from generate_series(" INT64_FORMAT ", " INT64_FORMAT ") as aid",
- naccounts, (int64) i * naccounts, (int64) (i + 1) * naccounts - 1);
+ "from generate_series(" INT64_FORMAT ", "
+ INT64_FORMAT ") as aid",
+ naccounts, (int64) i * naccounts,
+ (int64) (i + 1) * naccounts - 1);
+ executeStatement(con, sql.data);
+
+ executeStatement(con, "commit");
+ }
+
+ termPQExpBuffer(&sql);
+}
+
+/*
+ * Generating data via INSERT .. SELECT .. FROM unnest
+ * One transaction per 'scale'
+ */
+static void
+generateDataInsertUnnest(PGconn *con)
+{
+ PQExpBufferData sql;
+
+ fprintf(stderr, "via INSERT .. SELECT unnest...\n");
+
+ initPQExpBuffer(&sql);
+
+ executeStatement(con, "begin");
+
+ /* truncate away any old data */
+ initTruncateTables(con);
+
+ executeStatement(con, "commit");
+
+ for (int s = 0; s < scale; s++)
+ {
+ executeStatement(con, "begin");
+
+ printfPQExpBuffer(&sql,
+ "insert into pgbench_branches(bid,bbalance) "
+ "values(%d, 0)", s + 1);
+ executeStatement(con, sql.data);
+
+ printfPQExpBuffer(&sql,
+ "insert into pgbench_tellers(tid, bid, tbalance) "
+ "select unnest(array_agg(s.i order by s.i)) as tid, "
+ "%d as bid, 0 as tbalance "
+ "from generate_series(%d, %d) as s(i)",
+ s + 1, s * ntellers + 1, (s + 1) * ntellers);
+ executeStatement(con, sql.data);
+
+ printfPQExpBuffer(&sql,
+ "with data as ("
+ " select generate_series(" INT64_FORMAT ", "
+ INT64_FORMAT ") as i) "
+ "insert into pgbench_accounts(aid, bid, "
+ "abalance, filler) "
+ "select unnest(aid), unnest(bid), 0 as abalance, "
+ "'' as filler "
+ "from (select array_agg(i+1) aid, "
+ "array_agg(i/%d + 1) bid from data)",
+ (int64) s * naccounts + 1,
+ (int64) (s + 1) * naccounts, naccounts);
executeStatement(con, sql.data);
executeStatement(con, "commit");
@@ -5258,6 +5374,32 @@ initGenerateDataServerSide(PGconn *con)
termPQExpBuffer(&sql);
}
+/*
+ * Fill the standard tables with some data generated on the server
+ *
+ * As already the case with the client-side data generation, the filler
+ * column defaults to NULL in pgbench_branches and pgbench_tellers,
+ * and is a blank-padded string in pgbench_accounts.
+ */
+static void
+initGenerateDataServerSide(PGconn *con)
+{
+ fprintf(stderr, "generating data (server-side) ");
+
+ switch (data_generation_type)
+ {
+ case GEN_TYPE_INSERT_ORIGINAL:
+ generateDataInsertSingleTXN(con);
+ break;
+ case GEN_TYPE_INSERT_SERIES:
+ generateDataInsertSeries(con);
+ break;
+ case GEN_TYPE_INSERT_UNNEST:
+ generateDataInsertUnnest(con);
+ break;
+ }
+}
+
/*
* Invoke vacuum on the standard tables
*/
@@ -5341,6 +5483,8 @@ initCreateFKeys(PGconn *con)
static void
checkInitSteps(const char *initialize_steps)
{
+ char data_init_type = 0;
+
if (initialize_steps[0] == '\0')
pg_fatal("no initialization steps specified");
@@ -5352,7 +5496,26 @@ checkInitSteps(const char *initialize_steps)
pg_log_error_detail("Allowed step characters are: \"" ALL_INIT_STEPS "\".");
exit(1);
}
+
+ switch (*step)
+ {
+ case 'G':
+ data_init_type++;
+ data_generation_type = *step;
+ break;
+ case 'i':
+ data_init_type++;
+ data_generation_type = *step;
+ break;
+ case 'I':
+ data_init_type++;
+ data_generation_type = *step;
+ break;
+ }
}
+
+ if (data_init_type > 1)
+ pg_log_error("WARNING! More than one type of server-side data generation is requested");
}
/*
@@ -5395,6 +5558,8 @@ runInitSteps(const char *initialize_steps)
initGenerateDataClientSide(con);
break;
case 'G':
+ case 'i':
+ case 'I':
op = "server-side generate";
initGenerateDataServerSide(con);
break;
--
2.43.0
From 5e1827b889b283f50299ce6ab1a73f9f55a4a84f Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Mon, 10 Nov 2025 20:00:56 +0700
Subject: [PATCH v7 05/18] Fixing typo in query
---
src/bin/pgbench/pgbench.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 65d77cdefea..03e37df4434 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -5241,7 +5241,8 @@ generateDataInsertSingleTXN(PGconn *con)
printfPQExpBuffer(&sql,
"insert into pgbench_branches(bid, bbalance) "
"select bid, 0 "
- "from generate_series(1, %d)", scale * nbranches);
+ "from generate_series(1, %d) as bid",
+ scale * nbranches);
executeStatement(con, sql.data);
printfPQExpBuffer(&sql,
--
2.43.0
From 7ca86521fda6929b8e0de3fc77dcbb8984009c88 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Tue, 11 Nov 2025 19:39:45 +0700
Subject: [PATCH v7 06/18] Adding support for COPY BINARY mode
---
src/bin/pgbench/pgbench.c | 393 ++++++++++++++++++++++++++++++++++++--
1 file changed, 381 insertions(+), 12 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 03e37df4434..71aa1d9479f 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -161,7 +161,7 @@ typedef struct socket_set
* some configurable parameters */
#define DEFAULT_INIT_STEPS "dtgvp" /* default -I setting */
-#define ALL_INIT_STEPS "dtgGiIvpf" /* all possible steps */
+#define ALL_INIT_STEPS "dtgCGiIvpf" /* all possible steps */
#define LOG_STEP_SECONDS 5 /* seconds between log messages */
#define DEFAULT_NXACTS 10 /* default nxacts */
@@ -176,6 +176,8 @@ typedef struct socket_set
/* 'one transaction per scale' server-side methods */
#define GEN_TYPE_INSERT_SERIES 'i' /* use INSERT .. SELECT generate_series to generate data */
#define GEN_TYPE_INSERT_UNNEST 'I' /* use INSERT .. SELECT unnest to generate data */
+#define GEN_TYPE_COPY_ORIGINAL 'g' /* use COPY .. FROM STDIN .. TEXT to generate data */
+#define GEN_TYPE_COPY_BINARY 'C' /* use COPY .. FROM STDIN .. BINARY to generate data */
static int nxacts = 0; /* number of transactions per client */
static int duration = 0; /* duration in seconds */
@@ -188,10 +190,17 @@ static int64 end_time = 0; /* when to stop in micro seconds, under -T */
static int scale = 1;
/*
- *
+ * mode of data generation to use
*/
static char data_generation_type = '?';
+/*
+ * COPY FROM BINARY execution buffer
+ */
+#define BIN_COPY_BUF_SIZE 102400 /* maximum buffer size for COPY FROM BINARY */
+static char *bin_copy_buffer = NULL; /* buffer for COPY FROM BINARY */
+static int32_t bin_copy_buffer_length = 0; /* current buffer size */
+
/*
* fillfactor. for example, fillfactor = 90 will use only 90 percent
* space during inserts and leave 10 percent free.
@@ -861,7 +870,8 @@ static int wait_on_socket_set(socket_set *sa, int64 usecs);
static bool socket_has_input(socket_set *sa, int fd, int idx);
/* callback used to build rows for COPY during data loading */
-typedef void (*initRowMethod) (PQExpBufferData *sql, int64 curr);
+typedef void (*initRowMethod) (PQExpBufferData *sql, int64 curr);
+typedef void (*initRowMethodBin) (PGconn *con, PGresult *res, int64_t curr, int32_t parent);
/* callback functions for our flex lexer */
static const PsqlScanCallbacks pgbench_callbacks = {
@@ -925,6 +935,7 @@ usage(void)
" d: drop any existing pgbench tables\n"
" t: create the tables used by the standard pgbench scenario\n"
" g: generate data, client-side\n"
+ " C: client-side (single TNX) COPY .. FROM STDIN .. BINARY\n"
" G: generate data, server-side in single transaction\n"
" i: server-side (multiple TXNs) INSERT .. SELECT generate_series\n"
" I: server-side (multiple TXNs) INSERT .. SELECT unnest\n"
@@ -5191,9 +5202,9 @@ initPopulateTable(PGconn *con, const char *table, int64 base,
* a blank-padded string in pgbench_accounts.
*/
static void
-initGenerateDataClientSide(PGconn *con)
+initGenerateDataClientSideText(PGconn *con)
{
- fprintf(stderr, "generating data (client-side)...\n");
+ fprintf(stderr, "TEXT mode...\n");
/*
* we do all of this in one transaction to enable the backend's
@@ -5209,12 +5220,373 @@ initGenerateDataClientSide(PGconn *con)
* already exist
*/
initPopulateTable(con, "pgbench_branches", nbranches, initBranch);
- initPopulateTable(con, "pgbench_tellers", ntellers, initTeller);
+ initPopulateTable(con, "pgbench_tellers", ntellers, initTeller);
initPopulateTable(con, "pgbench_accounts", naccounts, initAccount);
executeStatement(con, "commit");
}
+
+/*
+ * Dumps binary buffer to file (purely for debugging)
+ */
+static void
+dumpBufferToFile(char *filename)
+{
+ FILE *file_ptr;
+ size_t bytes_written;
+
+ file_ptr = fopen(filename, "wb");
+ if (file_ptr == NULL)
+ {
+ fprintf(stderr, "Error opening file %s\n", filename);
+ return; // EXIT_FAILURE;
+ }
+
+ bytes_written = fwrite(bin_copy_buffer, 1, bin_copy_buffer_length, file_ptr);
+
+ if (bytes_written != bin_copy_buffer_length)
+ {
+ fprintf(stderr, "Error writing to file or incomplete write\n");
+ fclose(file_ptr);
+ return; // EXIT_FAILURE;
+ }
+
+ fclose(file_ptr);
+}
+
+/*
+ * Save char data to buffer
+ */
+static void
+bufferCharData(char *src, int32_t len)
+{
+ memcpy((char *) bin_copy_buffer + bin_copy_buffer_length, (char *) src, len);
+ bin_copy_buffer_length += len;
+}
+
+/*
+ * Converts platform byte order into network byte order
+ * SPARC doesn't reqire that
+ */
+static void
+bufferData(void *src, int32_t len)
+{
+#ifdef __sparc__
+ bufferCharData(src, len);
+#else
+ if (len == 1)
+ bufferCharData(src, len);
+ else
+ for (int32_t i = 0; i < len; i++)
+ {
+ ((char *) bin_copy_buffer + bin_copy_buffer_length)[i] =
+ ((char *) src)[len - i - 1];
+ }
+
+ bin_copy_buffer_length += len;
+#endif
+}
+
+/*
+ * adds column counter
+ */
+static void
+addColumnCounter(int16_t n)
+{
+ bufferData((void *) &n, sizeof(n));
+}
+
+/*
+ * adds column with NULL value
+ */
+static void
+addNullColumn()
+{
+ int32_t null = -1;
+ bufferData((void *) &null, sizeof(null));
+}
+
+/*
+ * adds column with int8 value
+ */
+static void
+addInt8Column(int8_t value)
+{
+ int8_t data = value;
+ int32_t size = sizeof(data);
+ bufferData((void *) &size, sizeof(size));
+ bufferData((void *) &data, sizeof(data));
+}
+
+/*
+ * adds column with int16 value
+ */
+static void
+addInt16Column(int16_t value)
+{
+ int16_t data = value;
+ int32_t size = sizeof(data);
+ bufferData((void *) &size, sizeof(size));
+ bufferData((void *) &data, sizeof(data));
+}
+
+/*
+ * adds column with inti32 value
+ */
+static void
+addInt32Column(int32_t value)
+{
+ int32_t data = value;
+ int32_t size = sizeof(data);
+ bufferData((void *) &size, sizeof(size));
+ bufferData((void *) &data, sizeof(data));
+}
+
+/*
+ * adds column with inti64 value
+ */
+static void
+addInt64Column(int64_t value)
+{
+ int64_t data = value;
+ int32_t size = sizeof(data);
+ bufferData((void *) &size, sizeof(size));
+ bufferData((void *) &data, sizeof(data));
+}
+
+/*
+ * adds column with char value
+ */
+static void
+addCharColumn(char *value)
+{
+ int32_t size = strlen(value);
+ bufferData((void *) &size, sizeof(size));
+ bufferCharData(value, size);
+}
+
+/*
+ * Starts communication with server for COPY FROM BINARY statement
+ */
+static void
+sendBinaryCopyHeader(PGconn *con)
+{
+ char header[] = {'P','G','C','O','P','Y','\n','\377','\r','\n','\0',
+ '\0','\0','\0','\0',
+ '\0','\0','\0','\0' };
+
+ PQputCopyData(con, header, 19);
+}
+
+/*
+ * Finishes communication with server for COPY FROM BINARY statement
+ */
+static void
+sendBinaryCopyTrailer(PGconn *con)
+{
+ static char trailer[] = { 0xFF, 0xFF };
+
+ PQputCopyData(con, trailer, 2);
+}
+
+/*
+ * Flashes current buffer over network if needed
+ */
+static void
+flushBuffer(PGconn *con, PGresult *res, int16_t row_len)
+{
+ if (bin_copy_buffer_length + row_len > BIN_COPY_BUF_SIZE)
+ {
+ /* flush current buffer */
+ if (PQresultStatus(res) == PGRES_COPY_IN)
+ PQputCopyData(con, (char *) bin_copy_buffer, bin_copy_buffer_length);
+ bin_copy_buffer_length = 0;
+ }
+}
+
+/*
+ * Sends current branch row to buffer
+ */
+static void
+initBranchBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
+{
+ /*
+ * Each row has following extra bytes:
+ * - 2 bytes for number of columns
+ * - 4 bytes as length for each column
+ */
+ int16_t max_row_len = 35 + 2 + 4*3; /* max row size is 32 */
+
+ flushBuffer(con, res, max_row_len);
+
+ addColumnCounter(2);
+
+ addInt32Column(curr + 1);
+ addInt32Column(0);
+}
+
+/*
+ * Sends current teller row to buffer
+ */
+static void
+initTellerBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
+{
+ /*
+ * Each row has following extra bytes:
+ * - 2 bytes for number of columns
+ * - 4 bytes as length for each column
+ */
+ int16_t max_row_len = 40 + 2 + 4*4; /* max row size is 40 */
+
+ flushBuffer(con, res, max_row_len);
+
+ addColumnCounter(3);
+
+ addInt32Column(curr + 1);
+ addInt32Column(curr / parent + 1);
+ addInt32Column(0);
+}
+
+/*
+ * Sends current account row to buffer
+ */
+static void
+initAccountBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
+{
+ /*
+ * Each row has following extra bytes:
+ * - 2 bytes for number of columns
+ * - 4 bytes as length for each column
+ */
+ int16_t max_row_len = 250 + 2 + 4*4; /* max row size is 250 for int64 */
+
+ flushBuffer(con, res, max_row_len);
+
+ addColumnCounter(3);
+
+ if (scale <= SCALE_32BIT_THRESHOLD)
+ addInt32Column(curr + 1);
+ else
+ addInt64Column(curr);
+
+ addInt32Column(curr / parent + 1);
+ addInt32Column(0);
+}
+
+/*
+ * Universal wrapper for sending data in binary format
+ */
+static void
+initPopulateTableBinary(PGconn *con, char *table, char *columns,
+ int64_t base, initRowMethodBin init_row)
+{
+ int n;
+ PGresult *res;
+ char copy_statement[256];
+ const char *copy_statement_fmt = "copy %s (%s) from stdin (format binary)";
+ int64_t total = base * scale;
+
+ bin_copy_buffer_length = 0;
+
+ /* Use COPY with FREEZE on v14 and later for all ordinary tables */
+ if ((PQserverVersion(con) >= 140000) &&
+ get_table_relkind(con, table) == RELKIND_RELATION)
+ copy_statement_fmt = "copy %s (%s) from stdin with (format binary, freeze on)";
+
+ n = pg_snprintf(copy_statement, sizeof(copy_statement), copy_statement_fmt, table, columns);
+ if (n >= sizeof(copy_statement))
+ pg_fatal("invalid buffer size: must be at least %d characters long", n);
+ else if (n == -1)
+ pg_fatal("invalid format string");
+
+ res = PQexec(con, copy_statement);
+
+ if (PQresultStatus(res) != PGRES_COPY_IN)
+ pg_fatal("unexpected copy in result: %s", PQerrorMessage(con));
+ PQclear(res);
+
+
+ sendBinaryCopyHeader(con);
+
+ for (int64_t i = 0; i < total; i++)
+ {
+ init_row(con, res, i, base);
+ }
+
+ if (PQresultStatus(res) == PGRES_COPY_IN)
+ PQputCopyData(con, (char *) bin_copy_buffer, bin_copy_buffer_length);
+ else
+ fprintf(stderr, "Unexpected mode %d instead of %d\n", PQresultStatus(res), PGRES_COPY_IN);
+
+ sendBinaryCopyTrailer(con);
+
+ if (PQresultStatus(res) == PGRES_COPY_IN)
+ {
+ if (PQputCopyEnd(con, NULL) == 1) /* success */
+ {
+ res = PQgetResult(con);
+ if (PQresultStatus(res) != PGRES_COMMAND_OK)
+ fprintf(stderr, "Error: %s\n", PQerrorMessage(con));
+ PQclear(res);
+ }
+ else
+ fprintf(stderr, "Error: %s\n", PQerrorMessage(con));
+ }
+}
+
+/*
+ * Wrapper for binary data load
+ */
+static void
+initGenerateDataClientSideBinary(PGconn *con)
+{
+
+ fprintf(stderr, "BINARY mode...\n");
+
+ bin_copy_buffer = pg_malloc(BIN_COPY_BUF_SIZE);
+ bin_copy_buffer_length = 0;
+
+ /*
+ * we do all of this in one transaction to enable the backend's
+ * data-loading optimizations
+ */
+ executeStatement(con, "begin");
+
+ /* truncate away any old data */
+ initTruncateTables(con);
+
+ initPopulateTableBinary(con, "pgbench_branches", "bid, bbalance",
+ nbranches, initBranchBinary);
+ initPopulateTableBinary(con, "pgbench_tellers", "tid, bid, tbalance",
+ ntellers, initTellerBinary);
+ initPopulateTableBinary(con, "pgbench_accounts", "aid, bid, abalance",
+ naccounts, initAccountBinary);
+
+ executeStatement(con, "commit");
+
+ pg_free(bin_copy_buffer);
+}
+
+/*
+ * Fill the standard tables with some data generated and sent from the client.
+ */
+static void
+initGenerateDataClientSide(PGconn *con)
+{
+ fprintf(stderr, "generating data (client-side) in ");
+
+ switch (data_generation_type)
+ {
+ case GEN_TYPE_COPY_ORIGINAL:
+ initGenerateDataClientSideText(con);
+ break;
+ case GEN_TYPE_COPY_BINARY:
+ initGenerateDataClientSideBinary(con);
+ break;
+ }
+}
+
/*
* Generating data via INSERT .. SELECT .. FROM generate_series
* whole dataset in single transaction
@@ -5500,14 +5872,10 @@ checkInitSteps(const char *initialize_steps)
switch (*step)
{
+ case 'g':
+ case 'C':
case 'G':
- data_init_type++;
- data_generation_type = *step;
- break;
case 'i':
- data_init_type++;
- data_generation_type = *step;
- break;
case 'I':
data_init_type++;
data_generation_type = *step;
@@ -5555,6 +5923,7 @@ runInitSteps(const char *initialize_steps)
initCreateTables(con);
break;
case 'g':
+ case 'C':
op = "client-side generate";
initGenerateDataClientSide(con);
break;
--
2.43.0
From 4aa0ac05765edf6b5f0c13e18ac677287ce78206 Mon Sep 17 00:00:00 2001
From: Fujii Masao <[email protected]>
Date: Fri, 14 Nov 2025 22:40:39 +0900
Subject: [PATCH v7 07/18] pgbench: Fix assertion failure with multiple
\syncpipeline in pipeline mode.
Previously, when pgbench ran a custom script that triggered retriable errors
(e.g., deadlocks) followed by multiple \syncpipeline commands in pipeline mode,
the following assertion failure could occur:
Assertion failed: (res == ((void*)0)), function discardUntilSync, file pgbench.c, line 3594.
The issue was that discardUntilSync() assumed a pipeline sync result
(PGRES_PIPELINE_SYNC) would always be followed by either another sync result
or NULL. This assumption was incorrect: when multiple sync requests were sent,
a sync result could instead be followed by another result type. In such cases,
discardUntilSync() mishandled the results, leading to the assertion failure.
This commit fixes the issue by making discardUntilSync() correctly handle cases
where a pipeline sync result is followed by other result types. It now continues
discarding results until another pipeline sync followed by NULL is reached.
Backpatched to v17, where support for \syncpipeline command in pgbench was
introduced.
Author: Yugo Nagata <[email protected]>
Reviewed-by: Chao Li <[email protected]>
Reviewed-by: Fujii Masao <[email protected]>
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 17
---
src/bin/pgbench/pgbench.c | 39 ++++++++++++++++++++++++++++-----------
1 file changed, 28 insertions(+), 11 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index d8764ba6fe0..a425176ecdc 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -3563,14 +3563,18 @@ doRetry(CState *st, pg_time_usec_t *now)
}
/*
- * Read results and discard it until a sync point.
+ * Read and discard results until the last sync point.
*/
static int
discardUntilSync(CState *st)
{
bool received_sync = false;
- /* send a sync */
+ /*
+ * Send a Sync message to ensure at least one PGRES_PIPELINE_SYNC is
+ * received and to avoid an infinite loop, since all earlier ones may have
+ * already been received.
+ */
if (!PQpipelineSync(st->con))
{
pg_log_error("client %d aborted: failed to send a pipeline sync",
@@ -3578,29 +3582,42 @@ discardUntilSync(CState *st)
return 0;
}
- /* receive PGRES_PIPELINE_SYNC and null following it */
+ /*
+ * Continue reading results until the last sync point, i.e., until
+ * reaching null just after PGRES_PIPELINE_SYNC.
+ */
for (;;)
{
PGresult *res = PQgetResult(st->con);
+ if (PQstatus(st->con) == CONNECTION_BAD)
+ {
+ pg_log_error("client %d aborted while rolling back the transaction after an error; perhaps the backend died while processing",
+ st->id);
+ PQclear(res);
+ return 0;
+ }
+
if (PQresultStatus(res) == PGRES_PIPELINE_SYNC)
received_sync = true;
- else if (received_sync)
+ else if (received_sync && res == NULL)
{
- /*
- * PGRES_PIPELINE_SYNC must be followed by another
- * PGRES_PIPELINE_SYNC or NULL; otherwise, assert failure.
- */
- Assert(res == NULL);
-
/*
* Reset ongoing sync count to 0 since all PGRES_PIPELINE_SYNC
* results have been discarded.
*/
st->num_syncs = 0;
- PQclear(res);
break;
}
+ else
+ {
+ /*
+ * If a PGRES_PIPELINE_SYNC is followed by something other than
+ * PGRES_PIPELINE_SYNC or NULL, another PGRES_PIPELINE_SYNC will
+ * appear later. Reset received_sync to false to wait for it.
+ */
+ received_sync = false;
+ }
PQclear(res);
}
--
2.43.0
From 9c4f19055597e9adb25e65c2aa8bedf20a09e13d Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Fri, 21 Nov 2025 19:05:58 +0700
Subject: [PATCH v7 08/18] Setting empty string as default value in filler
column
---
src/bin/pgbench/pgbench.c | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 967f6ce6984..03b5e5c28f0 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -4985,26 +4985,26 @@ initCreateTables(PGconn *con)
static const struct ddlinfo DDLs[] = {
{
"pgbench_history",
- "tid int,bid int,aid int,delta int,mtime timestamp,filler char(22)",
- "tid int,bid int,aid bigint,delta int,mtime timestamp,filler char(22)",
+ "tid int,bid int,aid int,delta int,mtime timestamp,filler char(22) default ''",
+ "tid int,bid int,aid bigint,delta int,mtime timestamp,filler char(22) default ''",
0
},
{
"pgbench_tellers",
- "tid int not null,bid int,tbalance int,filler char(84)",
- "tid int not null,bid int,tbalance int,filler char(84)",
+ "tid int not null,bid int,tbalance int,filler char(84) default ''",
+ "tid int not null,bid int,tbalance int,filler char(84) default ''",
1
},
{
"pgbench_accounts",
- "aid int not null,bid int,abalance int,filler char(84)",
- "aid bigint not null,bid int,abalance int,filler char(84)",
+ "aid int not null,bid int,abalance int,filler char(84) default ''",
+ "aid bigint not null,bid int,abalance int,filler char(84) default ''",
1
},
{
"pgbench_branches",
- "bid int not null,bbalance int,filler char(88)",
- "bid int not null,bbalance int,filler char(88)",
+ "bid int not null,bbalance int,filler char(88) default ''",
+ "bid int not null,bbalance int,filler char(88) default ''",
1
}
};
--
2.43.0
From 2aabaa52dffdb78fbefaef95163881c15e18ef29 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <[email protected]>
Date: Fri, 21 Nov 2025 15:03:11 +0200
Subject: [PATCH v7 09/18] Use strtoi64() in pgbench, replacing its open-coded
implementation
Makes the code a little simpler.
The old implementation accepted trailing whitespace, but that was
unnecessary. Firstly, its sibling function for parsing decimals,
strtodouble(), does not accept trailing whitespace. Secondly, none of
the callers can pass a string with trailing whitespace to it.
In the passing, check specifically for ERANGE before printing the "out
of range" error. On some systems, strtoul() and strtod() return EINVAL
on an empty or all-spaces string, and "invalid input syntax" is more
appropriate for that than "out of range". For the existing
strtodouble() function this is purely academical because it's never
called with errorOK==false, but let's be tidy. (Perhaps we should
remove the dead codepaths altogether, but I'll leave that for another
day.)
Reviewed-by: Chao Li <[email protected]>
Reviewed-by: Yuefei Shi <[email protected]>
Reviewed-by: Neil Chen <[email protected]>
Discussion: https://www.postgresql.org/message-id/[email protected]
---
src/bin/pgbench/pgbench.c | 83 +++++++++------------------------------
1 file changed, 19 insertions(+), 64 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index a425176ecdc..68774a59efd 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -982,13 +982,17 @@ usage(void)
progname, progname, PACKAGE_BUGREPORT, PACKAGE_NAME, PACKAGE_URL);
}
-/* return whether str matches "^\s*[-+]?[0-9]+$" */
+/*
+ * Return whether str matches "^\s*[-+]?[0-9]+$"
+ *
+ * This should agree with strtoint64() on what's accepted, ignoring overflows.
+ */
static bool
is_an_int(const char *str)
{
const char *ptr = str;
- /* skip leading spaces; cast is consistent with strtoint64 */
+ /* skip leading spaces */
while (*ptr && isspace((unsigned char) *ptr))
ptr++;
@@ -1012,9 +1016,6 @@ is_an_int(const char *str)
/*
* strtoint64 -- convert a string to 64-bit integer
*
- * This function is a slightly modified version of pg_strtoint64() from
- * src/backend/utils/adt/numutils.c.
- *
* The function returns whether the conversion worked, and if so
* "*result" is set to the result.
*
@@ -1023,71 +1024,25 @@ is_an_int(const char *str)
bool
strtoint64(const char *str, bool errorOK, int64 *result)
{
- const char *ptr = str;
- int64 tmp = 0;
- bool neg = false;
-
- /*
- * Do our own scan, rather than relying on sscanf which might be broken
- * for long long.
- *
- * As INT64_MIN can't be stored as a positive 64 bit integer, accumulate
- * value as a negative number.
- */
-
- /* skip leading spaces */
- while (*ptr && isspace((unsigned char) *ptr))
- ptr++;
-
- /* handle sign */
- if (*ptr == '-')
- {
- ptr++;
- neg = true;
- }
- else if (*ptr == '+')
- ptr++;
+ char *end;
- /* require at least one digit */
- if (unlikely(!isdigit((unsigned char) *ptr)))
- goto invalid_syntax;
+ errno = 0;
+ *result = strtoi64(str, &end, 10);
- /* process digits */
- while (*ptr && isdigit((unsigned char) *ptr))
+ if (unlikely(errno == ERANGE))
{
- int8 digit = (*ptr++ - '0');
-
- if (unlikely(pg_mul_s64_overflow(tmp, 10, &tmp)) ||
- unlikely(pg_sub_s64_overflow(tmp, digit, &tmp)))
- goto out_of_range;
+ if (!errorOK)
+ pg_log_error("value \"%s\" is out of range for type bigint", str);
+ return false;
}
- /* allow trailing whitespace, but not other trailing chars */
- while (*ptr != '\0' && isspace((unsigned char) *ptr))
- ptr++;
-
- if (unlikely(*ptr != '\0'))
- goto invalid_syntax;
-
- if (!neg)
+ if (unlikely(errno != 0 || end == str || *end != '\0'))
{
- if (unlikely(tmp == PG_INT64_MIN))
- goto out_of_range;
- tmp = -tmp;
+ if (!errorOK)
+ pg_log_error("invalid input syntax for type bigint: \"%s\"", str);
+ return false;
}
-
- *result = tmp;
return true;
-
-out_of_range:
- if (!errorOK)
- pg_log_error("value \"%s\" is out of range for type bigint", str);
- return false;
-
-invalid_syntax:
- if (!errorOK)
- pg_log_error("invalid input syntax for type bigint: \"%s\"", str);
- return false;
}
/* convert string to double, detecting overflows/underflows */
@@ -1099,14 +1054,14 @@ strtodouble(const char *str, bool errorOK, double *dv)
errno = 0;
*dv = strtod(str, &end);
- if (unlikely(errno != 0))
+ if (unlikely(errno == ERANGE))
{
if (!errorOK)
pg_log_error("value \"%s\" is out of range for type double", str);
return false;
}
- if (unlikely(end == str || *end != '\0'))
+ if (unlikely(errno != 0 || end == str || *end != '\0'))
{
if (!errorOK)
pg_log_error("invalid input syntax for type double: \"%s\"", str);
--
2.43.0
From dcb85d26f8132eaaf9d096e814b9bda49db7d478 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Fri, 21 Nov 2025 20:06:24 +0700
Subject: [PATCH v7 10/18] Switching COPY FROM BINARY ti run in multiple
transactions
---
src/bin/pgbench/pgbench.c | 27 ++++++++++++++++-----------
1 file changed, 16 insertions(+), 11 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 03b5e5c28f0..6b89007a63b 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -5496,20 +5496,20 @@ initAccountBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
*/
static void
initPopulateTableBinary(PGconn *con, char *table, char *columns,
- int64_t base, initRowMethodBin init_row)
+ int counter, int64_t base, initRowMethodBin init_row)
{
int n;
PGresult *res;
char copy_statement[256];
const char *copy_statement_fmt = "copy %s (%s) from stdin (format binary)";
- int64_t total = base * scale;
+ int64_t start = base * counter;
bin_copy_buffer_length = 0;
/* Use COPY with FREEZE on v14 and later for all ordinary tables */
if ((PQserverVersion(con) >= 140000) &&
get_table_relkind(con, table) == RELKIND_RELATION)
- copy_statement_fmt = "copy %s (%s) from stdin with (format binary, freeze on)";
+ copy_statement_fmt = "copy %s (%s) from stdin with (format binary)";
n = pg_snprintf(copy_statement, sizeof(copy_statement), copy_statement_fmt, table, columns);
if (n >= sizeof(copy_statement))
@@ -5526,7 +5526,7 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
sendBinaryCopyHeader(con);
- for (int64_t i = 0; i < total; i++)
+ for (int64_t i = start; i < start + base; i++)
{
init_row(con, res, i, base);
}
@@ -5573,15 +5573,20 @@ initGenerateDataClientSideBinary(PGconn *con)
/* truncate away any old data */
initTruncateTables(con);
- initPopulateTableBinary(con, "pgbench_branches", "bid, bbalance",
- nbranches, initBranchBinary);
- initPopulateTableBinary(con, "pgbench_tellers", "tid, bid, tbalance",
- ntellers, initTellerBinary);
- initPopulateTableBinary(con, "pgbench_accounts", "aid, bid, abalance",
- naccounts, initAccountBinary);
-
executeStatement(con, "commit");
+ for (int i = 0; i < scale; i++)
+ {
+ initPopulateTableBinary(con, "pgbench_branches", "bid, bbalance",
+ i, nbranches, initBranchBinary);
+ initPopulateTableBinary(con, "pgbench_tellers", "tid, bid, tbalance",
+ i, ntellers, initTellerBinary);
+ initPopulateTableBinary(con, "pgbench_accounts", "aid, bid, abalance",
+ i, naccounts, initAccountBinary);
+
+ executeStatement(con, "commit");
+ }
+
pg_free(bin_copy_buffer);
}
--
2.43.0
From b8e28881225234fd00b55235bc60fad2dc60b544 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Sat, 22 Nov 2025 17:06:00 +0700
Subject: [PATCH v7 11/18] Adding tests for new modes of data generation
---
src/bin/pgbench/pgbench.c | 21 ++++----
src/bin/pgbench/t/001_pgbench_with_server.pl | 52 +++++++++++++++++---
2 files changed, 56 insertions(+), 17 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 6b89007a63b..dd4e5d5e056 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -164,7 +164,7 @@ typedef struct socket_set
#define ALL_INIT_STEPS "dtgCGiIvpf" /* all possible steps */
#define LOG_STEP_SECONDS 5 /* seconds between log messages */
-#define DEFAULT_NXACTS 10 /* default nxacts */
+#define DEFAULT_NXACTS 10 /* default nxacts */
#define MIN_GAUSSIAN_PARAM 2.0 /* minimum parameter for gauss */
@@ -192,7 +192,7 @@ static int scale = 1;
/*
* mode of data generation to use
*/
-static char data_generation_type = '?';
+static char data_generation_type = GEN_TYPE_COPY_ORIGINAL;
/*
* COPY FROM BINARY execution buffer
@@ -4985,26 +4985,26 @@ initCreateTables(PGconn *con)
static const struct ddlinfo DDLs[] = {
{
"pgbench_history",
- "tid int,bid int,aid int,delta int,mtime timestamp,filler char(22) default ''",
- "tid int,bid int,aid bigint,delta int,mtime timestamp,filler char(22) default ''",
+ "tid int,bid int,aid int,delta int,mtime timestamp,filler char(22) default '?'",
+ "tid int,bid int,aid bigint,delta int,mtime timestamp,filler char(22) default '?'",
0
},
{
"pgbench_tellers",
- "tid int not null,bid int,tbalance int,filler char(84) default ''",
- "tid int not null,bid int,tbalance int,filler char(84) default ''",
+ "tid int not null,bid int,tbalance int,filler char(84)",
+ "tid int not null,bid int,tbalance int,filler char(84)",
1
},
{
"pgbench_accounts",
- "aid int not null,bid int,abalance int,filler char(84) default ''",
- "aid bigint not null,bid int,abalance int,filler char(84) default ''",
+ "aid int not null,bid int,abalance int,filler char(84) default '?'",
+ "aid bigint not null,bid int,abalance int,filler char(84) default '?'",
1
},
{
"pgbench_branches",
- "bid int not null,bbalance int,filler char(88) default ''",
- "bid int not null,bbalance int,filler char(88) default ''",
+ "bid int not null,bbalance int,filler char(88)",
+ "bid int not null,bbalance int,filler char(88)",
1
}
};
@@ -7837,6 +7837,7 @@ main(int argc, char **argv)
}
}
+ checkInitSteps(initialize_steps);
runInitSteps(initialize_steps);
exit(0);
}
diff --git a/src/bin/pgbench/t/001_pgbench_with_server.pl b/src/bin/pgbench/t/001_pgbench_with_server.pl
index 581e9af7907..a377048ead1 100644
--- a/src/bin/pgbench/t/001_pgbench_with_server.pl
+++ b/src/bin/pgbench/t/001_pgbench_with_server.pl
@@ -16,25 +16,30 @@ sub check_data_state
local $Test::Builder::Level = $Test::Builder::Level + 1;
my $node = shift;
my $type = shift;
+ my $sql_result;
- my $sql_result = $node->safe_psql('postgres',
- 'SELECT count(*) AS null_count FROM pgbench_accounts WHERE filler IS NULL LIMIT 10;'
- );
- is($sql_result, '0',
- "$type: filler column of pgbench_accounts has no NULL data");
$sql_result = $node->safe_psql('postgres',
'SELECT count(*) AS null_count FROM pgbench_branches WHERE filler IS NULL;'
);
is($sql_result, '1',
"$type: filler column of pgbench_branches has only NULL data");
+
$sql_result = $node->safe_psql('postgres',
'SELECT count(*) AS null_count FROM pgbench_tellers WHERE filler IS NULL;'
);
is($sql_result, '10',
"$type: filler column of pgbench_tellers has only NULL data");
+
+ $sql_result = $node->safe_psql('postgres',
+ 'SELECT count(*) AS null_count FROM pgbench_accounts WHERE filler IS NULL LIMIT 10;'
+ );
+ is($sql_result, '0',
+ "$type: filler column of pgbench_accounts has no NULL data");
+
$sql_result = $node->safe_psql('postgres',
'SELECT count(*) AS data_count FROM pgbench_history;');
- is($sql_result, '0', "$type: pgbench_history has no data");
+ is($sql_result, '0',
+ "$type: pgbench_history has no data");
}
# start a pgbench specific server
@@ -125,7 +130,7 @@ $node->pgbench(
'pgbench scale 1 initialization',);
# Check data state, after client-side data generation.
-check_data_state($node, 'client-side');
+check_data_state($node, 'client-side (default options)');
# Again, with all possible options
$node->pgbench(
@@ -143,6 +148,7 @@ $node->pgbench(
qr{done in \d+\.\d\d s }
],
'pgbench scale 1 initialization');
+check_data_state($node, 'client-side (all options)');
# Test interaction of --init-steps with legacy step-selection options
$node->pgbench(
@@ -164,6 +170,38 @@ $node->pgbench(
# Check data state, after server-side data generation.
check_data_state($node, 'server-side');
+# Test server-side generation with UNNEST
+$node->pgbench(
+ '--initialize --init-steps=dtI',
+ 0,
+ [qr{^$}],
+ [
+ qr{dropping old tables},
+ qr{creating tables},
+ qr{generating data \(server-side\)},
+ qr{done in \d+\.\d\d s }
+ ],
+ 'pgbench --init-steps server-side UNNEST');
+
+# Check data state, after server-side data generation.
+check_data_state($node, 'server-side (unnest)');
+
+# Test server-side generation with UNNEST
+$node->pgbench(
+ '--initialize --init-steps=dtC',
+ 0,
+ [qr{^$}],
+ [
+ qr{dropping old tables},
+ qr{creating tables},
+ qr{generating data \(client-side\)},
+ qr{done in \d+\.\d\d s }
+ ],
+ 'pgbench --init-steps client-side BINARY');
+
+# Check data state, after server-side data generation.
+check_data_state($node, 'client-side (binary)');
+
# Run all builtin scripts, for a few transactions each
$node->pgbench(
'--transactions=5 -Dfoo=bla --client=2 --protocol=simple --builtin=t'
--
2.43.0
From b3f2bce34232d299abc7644a8579f0ce49c8c9d6 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Sun, 23 Nov 2025 14:05:59 +0700
Subject: [PATCH v7 12/18] Fixing compiler warnings about unused procedures by
removing or commenting them out as they might be needed a bit later
---
src/bin/pgbench/pgbench.c | 31 +++++--------------------------
1 file changed, 5 insertions(+), 26 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 0a3ba21dcc9..682db61ff61 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -5201,7 +5201,7 @@ initGenerateDataClientSideText(PGconn *con)
/*
* Dumps binary buffer to file (purely for debugging)
- */
+ *
static void
dumpBufferToFile(char *filename)
{
@@ -5226,6 +5226,7 @@ dumpBufferToFile(char *filename)
fclose(file_ptr);
}
+ */
/*
* Save char data to buffer
@@ -5271,37 +5272,14 @@ addColumnCounter(int16_t n)
/*
* adds column with NULL value
- */
+ *
static void
addNullColumn()
{
int32_t null = -1;
bufferData((void *) &null, sizeof(null));
}
-
-/*
- * adds column with int8 value
*/
-static void
-addInt8Column(int8_t value)
-{
- int8_t data = value;
- int32_t size = sizeof(data);
- bufferData((void *) &size, sizeof(size));
- bufferData((void *) &data, sizeof(data));
-}
-
-/*
- * adds column with int16 value
- */
-static void
-addInt16Column(int16_t value)
-{
- int16_t data = value;
- int32_t size = sizeof(data);
- bufferData((void *) &size, sizeof(size));
- bufferData((void *) &data, sizeof(data));
-}
/*
* adds column with inti32 value
@@ -5329,7 +5307,7 @@ addInt64Column(int64_t value)
/*
* adds column with char value
- */
+ *
static void
addCharColumn(char *value)
{
@@ -5337,6 +5315,7 @@ addCharColumn(char *value)
bufferData((void *) &size, sizeof(size));
bufferCharData(value, size);
}
+ */
/*
* Starts communication with server for COPY FROM BINARY statement
--
2.43.0
From 9ab7fe302ba7de40593e7cba8ea1ca3b876c1ea5 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Thu, 29 Jan 2026 19:22:16 +0700
Subject: [PATCH v7 13/18] Moving PQclear call to the end of procedure to avoid
access to previously released memory
---
src/bin/pgbench/pgbench.c | 27 +++++++++++++++++----------
1 file changed, 17 insertions(+), 10 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 682db61ff61..3687b65871e 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -5363,11 +5363,11 @@ static void
initBranchBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
{
/*
- * Each row has following extra bytes:
+ * Each row of branch table has following extra bytes:
* - 2 bytes for number of columns
- * - 4 bytes as length for each column
+ * - 4 bytes as length for each of table's 3 columns
*/
- int16_t max_row_len = 35 + 2 + 4*3; /* max row size is 32 */
+ int16_t max_row_len = 35 + 2 + 4*3; /* max row size is 35 */
flushBuffer(con, res, max_row_len);
@@ -5384,9 +5384,9 @@ static void
initTellerBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
{
/*
- * Each row has following extra bytes:
+ * Each row of tellers table has following extra bytes:
* - 2 bytes for number of columns
- * - 4 bytes as length for each column
+ * - 4 bytes as length for each of table's 4 columns
*/
int16_t max_row_len = 40 + 2 + 4*4; /* max row size is 40 */
@@ -5406,9 +5406,9 @@ static void
initAccountBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
{
/*
- * Each row has following extra bytes:
+ * Each row of accounts table has following extra bytes:
* - 2 bytes for number of columns
- * - 4 bytes as length for each column
+ * - 4 bytes as length for each of table's 4 columns
*/
int16_t max_row_len = 250 + 2 + 4*4; /* max row size is 250 for int64 */
@@ -5455,11 +5455,11 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
if (PQresultStatus(res) != PGRES_COPY_IN)
pg_fatal("unexpected copy in result: %s", PQerrorMessage(con));
- PQclear(res);
sendBinaryCopyHeader(con);
+
for (int64_t i = start; i < start + base; i++)
{
init_row(con, res, i, base);
@@ -5470,8 +5470,10 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
else
fprintf(stderr, "Unexpected mode %d instead of %d\n", PQresultStatus(res), PGRES_COPY_IN);
+
sendBinaryCopyTrailer(con);
+
if (PQresultStatus(res) == PGRES_COPY_IN)
{
if (PQputCopyEnd(con, NULL) == 1) /* success */
@@ -5484,6 +5486,8 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
else
fprintf(stderr, "Error: %s\n", PQerrorMessage(con));
}
+
+ PQclear(res);
}
/*
@@ -5499,8 +5503,9 @@ initGenerateDataClientSideBinary(PGconn *con)
bin_copy_buffer_length = 0;
/*
- * we do all of this in one transaction to enable the backend's
- * data-loading optimizations
+ * we do all of this in multiple transactions
+ * to minimize load on DB server and perhaps
+ * in future allow load in parallel sessions
*/
executeStatement(con, "begin");
@@ -5511,6 +5516,8 @@ initGenerateDataClientSideBinary(PGconn *con)
for (int i = 0; i < scale; i++)
{
+ executeStatement(con, "begin");
+
initPopulateTableBinary(con, "pgbench_branches", "bid, bbalance",
i, nbranches, initBranchBinary);
initPopulateTableBinary(con, "pgbench_tellers", "tid, bid, tbalance",
--
2.43.0
From 3d7b2c53b3fa45b8848369bfbc49a0c7e5854304 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Thu, 29 Jan 2026 19:47:26 +0700
Subject: [PATCH v7 14/18] Fixing error about freeing memory twice
---
src/bin/pgbench/pgbench.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 3687b65871e..6f42438d970 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -5462,6 +5462,7 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
for (int64_t i = start; i < start + base; i++)
{
+ res = PQgetResult(con);
init_row(con, res, i, base);
}
@@ -5473,7 +5474,7 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
sendBinaryCopyTrailer(con);
-
+ res = PQgetResult(con);
if (PQresultStatus(res) == PGRES_COPY_IN)
{
if (PQputCopyEnd(con, NULL) == 1) /* success */
@@ -5481,7 +5482,6 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
res = PQgetResult(con);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
fprintf(stderr, "Error: %s\n", PQerrorMessage(con));
- PQclear(res);
}
else
fprintf(stderr, "Error: %s\n", PQerrorMessage(con));
--
2.43.0
From a71c90f556ba143335d9631ebbdf96b7120c6d1d Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Thu, 29 Jan 2026 20:24:54 +0700
Subject: [PATCH v7 15/18] Fixing memory leak during data flush in COPY BINARY
test
---
src/bin/pgbench/pgbench.c | 26 ++++++++++++++++----------
1 file changed, 16 insertions(+), 10 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 6f42438d970..8eca537b718 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -871,7 +871,7 @@ static bool socket_has_input(socket_set *sa, int fd, int idx);
/* callback used to build rows for COPY during data loading */
typedef void (*initRowMethod) (PQExpBufferData *sql, int64 curr);
-typedef void (*initRowMethodBin) (PGconn *con, PGresult *res, int64_t curr, int32_t parent);
+typedef void (*initRowMethodBin) (PGconn *con, int64_t curr, int32_t parent);
/* callback functions for our flex lexer */
static const PsqlScanCallbacks pgbench_callbacks = {
@@ -5345,13 +5345,18 @@ sendBinaryCopyTrailer(PGconn *con)
* Flashes current buffer over network if needed
*/
static void
-flushBuffer(PGconn *con, PGresult *res, int16_t row_len)
+flushBuffer(PGconn *con, int16_t row_len)
{
+ PGresult *res;
+
if (bin_copy_buffer_length + row_len > BIN_COPY_BUF_SIZE)
{
+ res = PQgetResult(con);
/* flush current buffer */
if (PQresultStatus(res) == PGRES_COPY_IN)
PQputCopyData(con, (char *) bin_copy_buffer, bin_copy_buffer_length);
+ else
+ pg_fatal("It is NOT a COPY command that is currently running");
bin_copy_buffer_length = 0;
}
}
@@ -5360,7 +5365,7 @@ flushBuffer(PGconn *con, PGresult *res, int16_t row_len)
* Sends current branch row to buffer
*/
static void
-initBranchBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
+initBranchBinary(PGconn *con, int64_t curr, int32_t parent)
{
/*
* Each row of branch table has following extra bytes:
@@ -5369,7 +5374,7 @@ initBranchBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
*/
int16_t max_row_len = 35 + 2 + 4*3; /* max row size is 35 */
- flushBuffer(con, res, max_row_len);
+ flushBuffer(con, max_row_len);
addColumnCounter(2);
@@ -5381,7 +5386,7 @@ initBranchBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
* Sends current teller row to buffer
*/
static void
-initTellerBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
+initTellerBinary(PGconn *con, int64_t curr, int32_t parent)
{
/*
* Each row of tellers table has following extra bytes:
@@ -5390,7 +5395,7 @@ initTellerBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
*/
int16_t max_row_len = 40 + 2 + 4*4; /* max row size is 40 */
- flushBuffer(con, res, max_row_len);
+ flushBuffer(con, max_row_len);
addColumnCounter(3);
@@ -5403,7 +5408,7 @@ initTellerBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
* Sends current account row to buffer
*/
static void
-initAccountBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
+initAccountBinary(PGconn *con, int64_t curr, int32_t parent)
{
/*
* Each row of accounts table has following extra bytes:
@@ -5412,7 +5417,7 @@ initAccountBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
*/
int16_t max_row_len = 250 + 2 + 4*4; /* max row size is 250 for int64 */
- flushBuffer(con, res, max_row_len);
+ flushBuffer(con, max_row_len);
addColumnCounter(3);
@@ -5462,10 +5467,10 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
for (int64_t i = start; i < start + base; i++)
{
- res = PQgetResult(con);
- init_row(con, res, i, base);
+ init_row(con, i, base);
}
+ res = PQgetResult(con);
if (PQresultStatus(res) == PGRES_COPY_IN)
PQputCopyData(con, (char *) bin_copy_buffer, bin_copy_buffer_length);
else
@@ -5474,6 +5479,7 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
sendBinaryCopyTrailer(con);
+
res = PQgetResult(con);
if (PQresultStatus(res) == PGRES_COPY_IN)
{
--
2.43.0
From a5933f40be0f3d408bb920f296fb5119c3ec28f4 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Fri, 30 Jan 2026 13:59:36 +0700
Subject: [PATCH v7 16/18] Fixing memory leak shown by valgrind
---
src/bin/pgbench/pgbench.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 8eca537b718..466d9023d72 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -5357,6 +5357,8 @@ flushBuffer(PGconn *con, int16_t row_len)
PQputCopyData(con, (char *) bin_copy_buffer, bin_copy_buffer_length);
else
pg_fatal("It is NOT a COPY command that is currently running");
+
+ PQclear(res);
bin_copy_buffer_length = 0;
}
}
@@ -5460,6 +5462,7 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
if (PQresultStatus(res) != PGRES_COPY_IN)
pg_fatal("unexpected copy in result: %s", PQerrorMessage(con));
+ PQclear(res);
sendBinaryCopyHeader(con);
@@ -5475,6 +5478,7 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
PQputCopyData(con, (char *) bin_copy_buffer, bin_copy_buffer_length);
else
fprintf(stderr, "Unexpected mode %d instead of %d\n", PQresultStatus(res), PGRES_COPY_IN);
+ PQclear(res);
sendBinaryCopyTrailer(con);
@@ -5485,6 +5489,7 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
{
if (PQputCopyEnd(con, NULL) == 1) /* success */
{
+ PQclear(res);
res = PQgetResult(con);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
fprintf(stderr, "Error: %s\n", PQerrorMessage(con));
@@ -5492,7 +5497,6 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
else
fprintf(stderr, "Error: %s\n", PQerrorMessage(con));
}
-
PQclear(res);
}
--
2.43.0
From 6a3036f898b050d5e71e70eaceccd63003aa6444 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Fri, 6 Feb 2026 13:53:07 +0700
Subject: [PATCH v7 17/18] Adding ability to switch data init between single
and multiple transactions
---
src/bin/pgbench/pgbench.c | 192 +++++++++----------
src/bin/pgbench/t/001_pgbench_with_server.pl | 93 ++++++++-
2 files changed, 173 insertions(+), 112 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 466d9023d72..af71e358e71 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -159,9 +159,8 @@ typedef struct socket_set
/********************************************************************
* some configurable parameters */
-
-#define DEFAULT_INIT_STEPS "dtgvp" /* default -I setting */
-#define ALL_INIT_STEPS "dtgCGiIvpf" /* all possible steps */
+#define DEFAULT_INIT_STEPS "dtgvp" /* default -I setting */
+#define ALL_INIT_STEPS "dtgMScGUvpf" /* all possible steps */
#define LOG_STEP_SECONDS 5 /* seconds between log messages */
#define DEFAULT_NXACTS 10 /* default nxacts */
@@ -171,14 +170,17 @@ typedef struct socket_set
#define MIN_ZIPFIAN_PARAM 1.001 /* minimum parameter for zipfian */
#define MAX_ZIPFIAN_PARAM 1000.0 /* maximum parameter for zipfian */
-/* original single transaction server-side method */
-#define GEN_TYPE_INSERT_ORIGINAL 'G' /* use INSERT .. SELECT generate_series to generate data */
-/* 'one transaction per scale' server-side methods */
-#define GEN_TYPE_INSERT_SERIES 'i' /* use INSERT .. SELECT generate_series to generate data */
-#define GEN_TYPE_INSERT_UNNEST 'I' /* use INSERT .. SELECT unnest to generate data */
-#define GEN_TYPE_COPY_ORIGINAL 'g' /* use COPY .. FROM STDIN .. TEXT to generate data */
-#define GEN_TYPE_COPY_BINARY 'C' /* use COPY .. FROM STDIN .. BINARY to generate data */
-
+/* server-side methods to generate data */
+#define INIT_STEP_GEN_TYPE_INSERT_SERIES 'G' /* use INSERT .. SELECT generate_series to generate data */
+#define INIT_STEP_GEN_TYPE_INSERT_UNNEST 'U' /* use INSERT .. SELECT unnest to generate data */
+/* client-side methods to generate data */
+#define INIT_STEP_GEN_TYPE_COPY_TEXT 'g' /* use COPY .. FROM STDIN .. TEXT to generate data */
+#define INIT_STEP_GEN_TYPE_COPY_BINARY 'c' /* use COPY .. FROM STDIN .. BINARY to generate data */
+/* data init pseudo steps */
+#define INIT_STEP_GEN_TYPE_SINGLE_XACT 'S' /* switch to init data as single transaction */
+#define INIT_STEP_GEN_TYPE_MULTI_XACT 'M' /* switch to init data as multiple transactions */
+
+static bool multi_xact = false; /* init data type (as single or multiple transactions) */
static int nxacts = 0; /* number of transactions per client */
static int duration = 0; /* duration in seconds */
static int64 end_time = 0; /* when to stop in micro seconds, under -T */
@@ -192,7 +194,7 @@ static int scale = 1;
/*
* mode of data generation to use
*/
-static char data_generation_type = GEN_TYPE_COPY_ORIGINAL;
+static char data_generation_type = INIT_STEP_GEN_TYPE_COPY_TEXT;
/*
* COPY FROM BINARY execution buffer
@@ -934,11 +936,13 @@ usage(void)
" run selected initialization steps, in the specified order\n"
" d: drop any existing pgbench tables\n"
" t: create the tables used by the standard pgbench scenario\n"
- " g: generate data, client-side\n"
- " C: client-side (single TNX) COPY .. FROM STDIN .. BINARY\n"
- " G: generate data, server-side in single transaction\n"
- " i: server-side (multiple TXNs) INSERT .. SELECT generate_series\n"
- " I: server-side (multiple TXNs) INSERT .. SELECT unnest\n"
+ " to generate data, client-side:\n"
+ " g: COPY .. FROM STDIN .. TEXT\n"
+ " c: COPY .. FROM STDIN .. BINARY\n"
+ " to generate data, server-side:\n"
+ " G: INSERT .. SELECT generate_series\n"
+ " U: INSERT .. SELECT unnest\n"
+ " M: use multiple transactions to initialize data\n"
" v: invoke VACUUM on the standard tables\n"
" p: create primary key indexes on the standard tables\n"
" f: create foreign keys between the standard tables\n"
@@ -5049,8 +5053,8 @@ initAccount(PQExpBufferData *sql, int64 curr)
}
static void
-initPopulateTable(PGconn *con, const char *table, int64 base,
- initRowMethod init_row)
+initPopulateTableText(PGconn *con, const char *table, int64 base,
+ initRowMethod init_row)
{
int n;
int64 k;
@@ -5178,6 +5182,9 @@ initGenerateDataClientSideText(PGconn *con)
{
fprintf(stderr, "TEXT mode...\n");
+ if (multi_xact)
+ fprintf(stderr, "WARNING! Multiple transactions are not supported in this mode\n");
+
/*
* we do all of this in one transaction to enable the backend's
* data-loading optimizations
@@ -5191,9 +5198,9 @@ initGenerateDataClientSideText(PGconn *con)
* fill branches, tellers, accounts in that order in case foreign keys
* already exist
*/
- initPopulateTable(con, "pgbench_branches", nbranches, initBranch);
- initPopulateTable(con, "pgbench_tellers", ntellers, initTeller);
- initPopulateTable(con, "pgbench_accounts", naccounts, initAccount);
+ initPopulateTableText(con, "pgbench_branches", nbranches, initBranch);
+ initPopulateTableText(con, "pgbench_tellers", ntellers, initTeller);
+ initPopulateTableText(con, "pgbench_accounts", naccounts, initAccount);
executeStatement(con, "commit");
}
@@ -5450,7 +5457,8 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
/* Use COPY with FREEZE on v14 and later for all ordinary tables */
if ((PQserverVersion(con) >= 140000) &&
get_table_relkind(con, table) == RELKIND_RELATION)
- copy_statement_fmt = "copy %s (%s) from stdin with (format binary)";
+ if (!multi_xact)
+ copy_statement_fmt = "copy %s (%s) from stdin with (format binary, freeze on)";
n = pg_snprintf(copy_statement, sizeof(copy_statement), copy_statement_fmt, table, columns);
if (n >= sizeof(copy_statement))
@@ -5522,11 +5530,13 @@ initGenerateDataClientSideBinary(PGconn *con)
/* truncate away any old data */
initTruncateTables(con);
- executeStatement(con, "commit");
+ if (multi_xact)
+ executeStatement(con, "commit");
for (int i = 0; i < scale; i++)
{
- executeStatement(con, "begin");
+ if (multi_xact)
+ executeStatement(con, "begin");
initPopulateTableBinary(con, "pgbench_branches", "bid, bbalance",
i, nbranches, initBranchBinary);
@@ -5535,9 +5545,13 @@ initGenerateDataClientSideBinary(PGconn *con)
initPopulateTableBinary(con, "pgbench_accounts", "aid, bid, abalance",
i, naccounts, initAccountBinary);
- executeStatement(con, "commit");
+ if (multi_xact)
+ executeStatement(con, "commit");
}
+ if (!multi_xact)
+ executeStatement(con, "commit");
+
pg_free(bin_copy_buffer);
}
@@ -5547,14 +5561,15 @@ initGenerateDataClientSideBinary(PGconn *con)
static void
initGenerateDataClientSide(PGconn *con)
{
- fprintf(stderr, "generating data (client-side) in ");
+ fprintf(stderr, "generating data (client-side as %s transaction%s) in ",
+ multi_xact ? "multiple" : "single", multi_xact ? "s" : "");
switch (data_generation_type)
{
- case GEN_TYPE_COPY_ORIGINAL:
+ case INIT_STEP_GEN_TYPE_COPY_TEXT:
initGenerateDataClientSideText(con);
break;
- case GEN_TYPE_COPY_BINARY:
+ case INIT_STEP_GEN_TYPE_COPY_BINARY:
initGenerateDataClientSideBinary(con);
break;
}
@@ -5562,58 +5577,7 @@ initGenerateDataClientSide(PGconn *con)
/*
* Generating data via INSERT .. SELECT .. FROM generate_series
- * whole dataset in single transaction
- */
-static void
-generateDataInsertSingleTXN(PGconn *con)
-{
- PQExpBufferData sql;
-
- fprintf(stderr, "via INSERT .. SELECT generate_series... in single TXN\n");
-
-
- /*
- * we do all of this in one transaction to enable the backend's
- * data-loading optimizations
- */
- executeStatement(con, "begin");
-
- /* truncate away any old data */
- initTruncateTables(con);
-
- initPQExpBuffer(&sql);
-
- printfPQExpBuffer(&sql,
- "insert into pgbench_branches(bid, bbalance) "
- "select bid, 0 "
- "from generate_series(1, %d) as bid",
- scale * nbranches);
- executeStatement(con, sql.data);
-
- printfPQExpBuffer(&sql,
- "insert into pgbench_tellers(tid, bid, tbalance) "
- "select tid + 1, tid / %d + 1, 0 "
- "from generate_series(0, %d) as tid",
- ntellers, (scale * ntellers) - 1);
- executeStatement(con, sql.data);
-
- printfPQExpBuffer(&sql,
- "insert into pgbench_accounts(aid, bid, abalance, "
- "filler) "
- "select aid + 1, aid / %d + 1, 0, '' "
- "from generate_series(0, " INT64_FORMAT ") as aid",
- naccounts, (int64) (scale * naccounts) - 1);
- executeStatement(con, sql.data);
-
- executeStatement(con, "commit");
-
- termPQExpBuffer(&sql);
-}
-
-
-/*
- * Generating data via INSERT .. SELECT .. FROM generate_series
- * One transaction per 'scale'
+ * Possibly as "One transaction per scale" in multi-transaction mode
*/
static void
generateDataInsertSeries(PGconn *con)
@@ -5629,11 +5593,13 @@ generateDataInsertSeries(PGconn *con)
/* truncate away any old data */
initTruncateTables(con);
- executeStatement(con, "commit");
+ if (multi_xact)
+ executeStatement(con, "commit");
for (int i = 0; i < scale; i++)
{
- executeStatement(con, "begin");
+ if (multi_xact)
+ executeStatement(con, "begin");
printfPQExpBuffer(&sql,
"insert into pgbench_branches(bid, bbalance) "
@@ -5657,15 +5623,19 @@ generateDataInsertSeries(PGconn *con)
(int64) (i + 1) * naccounts - 1);
executeStatement(con, sql.data);
- executeStatement(con, "commit");
+ if (multi_xact)
+ executeStatement(con, "commit");
}
+ if (!multi_xact)
+ executeStatement(con, "commit");
+
termPQExpBuffer(&sql);
}
/*
* Generating data via INSERT .. SELECT .. FROM unnest
- * One transaction per 'scale'
+ * Possibly as "One transaction per scale" in multi-tansaction mode
*/
static void
generateDataInsertUnnest(PGconn *con)
@@ -5681,11 +5651,13 @@ generateDataInsertUnnest(PGconn *con)
/* truncate away any old data */
initTruncateTables(con);
- executeStatement(con, "commit");
+ if (multi_xact)
+ executeStatement(con, "commit");
for (int s = 0; s < scale; s++)
{
- executeStatement(con, "begin");
+ if (multi_xact)
+ executeStatement(con, "begin");
printfPQExpBuffer(&sql,
"insert into pgbench_branches(bid,bbalance) "
@@ -5714,14 +5686,18 @@ generateDataInsertUnnest(PGconn *con)
(int64) (s + 1) * naccounts, naccounts);
executeStatement(con, sql.data);
- executeStatement(con, "commit");
+ if (multi_xact)
+ executeStatement(con, "commit");
}
+ if (!multi_xact)
+ executeStatement(con, "commit");
+
termPQExpBuffer(&sql);
}
/*
- * Fill the standard tables with some data generated on the server
+ * Fill the standard tables with some data generated on the server side
*
* As already the case with the client-side data generation, the filler
* column defaults to NULL in pgbench_branches and pgbench_tellers,
@@ -5730,17 +5706,15 @@ generateDataInsertUnnest(PGconn *con)
static void
initGenerateDataServerSide(PGconn *con)
{
- fprintf(stderr, "generating data (server-side) ");
+ fprintf(stderr, "generating data (server-side as %s transaction%s) ",
+ multi_xact ? "multiple" : "single", multi_xact ? "s" : "");
switch (data_generation_type)
{
- case GEN_TYPE_INSERT_ORIGINAL:
- generateDataInsertSingleTXN(con);
- break;
- case GEN_TYPE_INSERT_SERIES:
+ case INIT_STEP_GEN_TYPE_INSERT_SERIES:
generateDataInsertSeries(con);
break;
- case GEN_TYPE_INSERT_UNNEST:
+ case INIT_STEP_GEN_TYPE_INSERT_UNNEST:
generateDataInsertUnnest(con);
break;
}
@@ -5845,19 +5819,20 @@ checkInitSteps(const char *initialize_steps)
switch (*step)
{
- case 'g':
- case 'C':
- case 'G':
- case 'i':
- case 'I':
+ case INIT_STEP_GEN_TYPE_COPY_TEXT:
+ case INIT_STEP_GEN_TYPE_COPY_BINARY:
+ case INIT_STEP_GEN_TYPE_INSERT_SERIES:
+ case INIT_STEP_GEN_TYPE_INSERT_UNNEST:
data_init_type++;
data_generation_type = *step;
break;
}
}
+ if (data_init_type == 0)
+ pg_log_error("WARNING! No data generation type is provided");
if (data_init_type > 1)
- pg_log_error("WARNING! More than one type of server-side data generation is requested");
+ pg_log_error("WARNING! More than one type of data initialization is requested");
}
/*
@@ -5895,17 +5870,22 @@ runInitSteps(const char *initialize_steps)
op = "create tables";
initCreateTables(con);
break;
- case 'g':
- case 'C':
+ case INIT_STEP_GEN_TYPE_COPY_TEXT:
+ case INIT_STEP_GEN_TYPE_COPY_BINARY:
op = "client-side generate";
initGenerateDataClientSide(con);
break;
- case 'G':
- case 'i':
- case 'I':
+ case INIT_STEP_GEN_TYPE_INSERT_SERIES:
+ case INIT_STEP_GEN_TYPE_INSERT_UNNEST:
op = "server-side generate";
initGenerateDataServerSide(con);
break;
+ case INIT_STEP_GEN_TYPE_SINGLE_XACT:
+ multi_xact = false;
+ break;
+ case INIT_STEP_GEN_TYPE_MULTI_XACT:
+ multi_xact = true;
+ break;
case 'v':
op = "vacuum";
initVacuum(con);
diff --git a/src/bin/pgbench/t/001_pgbench_with_server.pl b/src/bin/pgbench/t/001_pgbench_with_server.pl
index a377048ead1..acb1e31d3e8 100644
--- a/src/bin/pgbench/t/001_pgbench_with_server.pl
+++ b/src/bin/pgbench/t/001_pgbench_with_server.pl
@@ -117,6 +117,7 @@ $node->pgbench(
[qr{Perhaps you need to do initialization}],
'run without init');
+
# Initialize pgbench tables scale 1
$node->pgbench(
'-i', 0,
@@ -160,7 +161,7 @@ $node->pgbench(
qr{creating tables},
qr{creating 3 partitions},
qr{creating primary keys},
- qr{generating data \(server-side\)},
+ qr{generating data \(server-side as single transaction\)},
qr{creating foreign keys},
qr{(?!vacuuming)}, # no vacuum
qr{done in \d+\.\d\d s }
@@ -170,15 +171,16 @@ $node->pgbench(
# Check data state, after server-side data generation.
check_data_state($node, 'server-side');
+
# Test server-side generation with UNNEST
$node->pgbench(
- '--initialize --init-steps=dtI',
+ '--initialize --init-steps=dtU',
0,
[qr{^$}],
[
qr{dropping old tables},
qr{creating tables},
- qr{generating data \(server-side\)},
+ qr{generating data \(server-side as single transaction\)},
qr{done in \d+\.\d\d s }
],
'pgbench --init-steps server-side UNNEST');
@@ -186,15 +188,48 @@ $node->pgbench(
# Check data state, after server-side data generation.
check_data_state($node, 'server-side (unnest)');
-# Test server-side generation with UNNEST
+
+# Test server-side generation with COPY TEXT
+$node->pgbench(
+ '--initialize --init-steps=dtg',
+ 0,
+ [qr{^$}],
+ [
+ qr{dropping old tables},
+ qr{creating tables},
+ qr{generating data \(client-side as single transaction},
+ qr{done in \d+\.\d\d s }
+ ],
+ 'pgbench --init-steps client-side TEXT');
+
+# Check data state, after server-side data generation.
+check_data_state($node, 'client-side (text)');
+
+$node->pgbench(
+ '--initialize --init-steps=dtMg',
+ 0,
+ [qr{^$}],
+ [
+ qr{dropping old tables},
+ qr{creating tables},
+ qr{generating data \(client-side as multiple transactions},
+ qr{done in \d+\.\d\d s }
+ ],
+ 'pgbench --init-steps client-side TEXT');
+
+# Check data state, after server-side data generation.
+check_data_state($node, 'client-side (text)');
+
+
+# Test server-side generation with COPY BINARY
$node->pgbench(
- '--initialize --init-steps=dtC',
+ '--initialize --init-steps=dtc',
0,
[qr{^$}],
[
qr{dropping old tables},
qr{creating tables},
- qr{generating data \(client-side\)},
+ qr{generating data \(client-side as single transaction},
qr{done in \d+\.\d\d s }
],
'pgbench --init-steps client-side BINARY');
@@ -202,6 +237,52 @@ $node->pgbench(
# Check data state, after server-side data generation.
check_data_state($node, 'client-side (binary)');
+$node->pgbench(
+ '--initialize --init-steps=dtSc',
+ 0,
+ [qr{^$}],
+ [
+ qr{dropping old tables},
+ qr{creating tables},
+ qr{generating data \(client-side as single transaction},
+ qr{done in \d+\.\d\d s }
+ ],
+ 'pgbench --init-steps client-side BINARY');
+
+# Check data state, after server-side data generation.
+check_data_state($node, 'client-side (binary)');
+
+$node->pgbench(
+ '--initialize --init-steps=dtMc',
+ 0,
+ [qr{^$}],
+ [
+ qr{dropping old tables},
+ qr{creating tables},
+ qr{generating data \(client-side as multiple transactions},
+ qr{done in \d+\.\d\d s }
+ ],
+ 'pgbench --init-steps client-side BINARY');
+
+# Check data state, after server-side data generation.
+check_data_state($node, 'client-side (binary)');
+
+$node->pgbench(
+ '--initialize --init-steps=dtMc',
+ 0,
+ [qr{^$}],
+ [
+ qr{dropping old tables},
+ qr{creating tables},
+ qr{generating data \(client-side as multiple transactions},
+ qr{done in \d+\.\d\d s }
+ ],
+ 'pgbench --init-steps client-side BINARY');
+
+# Check data state, after server-side data generation.
+check_data_state($node, 'client-side (binary)');
+
+
# Run all builtin scripts, for a few transactions each
$node->pgbench(
'--transactions=5 -Dfoo=bla --client=2 --protocol=simple --builtin=t'
--
2.43.0
From 2471bb47bf715402ece61b084b69afe3bb53742e Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Fri, 6 Feb 2026 13:55:47 +0700
Subject: [PATCH v7 18/18] Removing commented out procedures
---
src/bin/pgbench/pgbench.c | 52 ---------------------------------------
1 file changed, 52 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index af71e358e71..34464b7037c 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -5206,35 +5206,6 @@ initGenerateDataClientSideText(PGconn *con)
}
-/*
- * Dumps binary buffer to file (purely for debugging)
- *
-static void
-dumpBufferToFile(char *filename)
-{
- FILE *file_ptr;
- size_t bytes_written;
-
- file_ptr = fopen(filename, "wb");
- if (file_ptr == NULL)
- {
- fprintf(stderr, "Error opening file %s\n", filename);
- return; // EXIT_FAILURE;
- }
-
- bytes_written = fwrite(bin_copy_buffer, 1, bin_copy_buffer_length, file_ptr);
-
- if (bytes_written != bin_copy_buffer_length)
- {
- fprintf(stderr, "Error writing to file or incomplete write\n");
- fclose(file_ptr);
- return; // EXIT_FAILURE;
- }
-
- fclose(file_ptr);
-}
- */
-
/*
* Save char data to buffer
*/
@@ -5277,17 +5248,6 @@ addColumnCounter(int16_t n)
bufferData((void *) &n, sizeof(n));
}
-/*
- * adds column with NULL value
- *
-static void
-addNullColumn()
-{
- int32_t null = -1;
- bufferData((void *) &null, sizeof(null));
-}
- */
-
/*
* adds column with inti32 value
*/
@@ -5312,18 +5272,6 @@ addInt64Column(int64_t value)
bufferData((void *) &data, sizeof(data));
}
-/*
- * adds column with char value
- *
-static void
-addCharColumn(char *value)
-{
- int32_t size = strlen(value);
- bufferData((void *) &size, sizeof(size));
- bufferCharData(value, size);
-}
- */
-
/*
* Starts communication with server for COPY FROM BINARY statement
*/
--
2.43.0
^ permalink raw reply [nested|flat] 98+ messages in thread
* Re: Idea to enhance pgbench by more modes to generate data (multi-TXNs, UNNEST, COPY BINARY)
2026-01-30 13:22 RE: Idea to enhance pgbench by more modes to generate data (multi-TXNs, UNNEST, COPY BINARY) Madyshev Egor <[email protected]>
2026-01-30 14:47 ` Re: Idea to enhance pgbench by more modes to generate data (multi-TXNs, UNNEST, COPY BINARY) Boris Mironov <[email protected]>
2026-02-06 10:48 ` Re: Idea to enhance pgbench by more modes to generate data (multi-TXNs, UNNEST, COPY BINARY) Boris Mironov <[email protected]>
@ 2026-02-06 11:08 ` Boris Mironov <[email protected]>
2026-02-09 13:19 ` RE: Idea to enhance pgbench by more modes to generate data (multi-TXNs, UNNEST, COPY BINARY) Madyshev Egor <[email protected]>
0 siblings, 1 reply; 98+ messages in thread
From: Boris Mironov @ 2026-02-06 11:08 UTC (permalink / raw)
To: Madyshev Egor <[email protected]>; pgsql-hackers
Resending patch as Excel spreadsheet took over second attachment in last email and testing bot got confused
Attachments:
[application/octet-stream] v7-pgbench-faster-modes.patch (81.3K, ../../PH0PR08MB70207B96C493E5777DB4D6428866A@PH0PR08MB7020.namprd08.prod.outlook.com/3-v7-pgbench-faster-modes.patch)
download | inline diff:
From c768f399c556295de7d53895410e686d86b4b960 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Sun, 9 Nov 2025 19:34:58 +0700
Subject: [PATCH v7 01/18] Converting one huge transaction into series of one
per 'scale'
---
src/bin/pgbench/pgbench.c | 61 ++++++++++++++++++++++++++-------------
1 file changed, 41 insertions(+), 20 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index d8764ba6fe0..284a7c860f1 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -181,6 +181,12 @@ static int64 end_time = 0; /* when to stop in micro seconds, under -T */
*/
static int scale = 1;
+/*
+ * scaling factor after which we switch to multiple transactions during
+ * data population phase on server side
+ */
+static int64 single_txn_scale_limit = 1;
+
/*
* fillfactor. for example, fillfactor = 90 will use only 90 percent
* space during inserts and leave 10 percent free.
@@ -5213,6 +5219,7 @@ static void
initGenerateDataServerSide(PGconn *con)
{
PQExpBufferData sql;
+ int chunk = (scale >= single_txn_scale_limit) ? 1 : scale;
fprintf(stderr, "generating data (server-side)...\n");
@@ -5225,30 +5232,44 @@ initGenerateDataServerSide(PGconn *con)
/* truncate away any old data */
initTruncateTables(con);
+ executeStatement(con, "commit");
+
initPQExpBuffer(&sql);
- printfPQExpBuffer(&sql,
- "insert into pgbench_branches(bid,bbalance) "
- "select bid, 0 "
- "from generate_series(1, %d) as bid", nbranches * scale);
- executeStatement(con, sql.data);
-
- printfPQExpBuffer(&sql,
- "insert into pgbench_tellers(tid,bid,tbalance) "
- "select tid, (tid - 1) / %d + 1, 0 "
- "from generate_series(1, %d) as tid", ntellers, ntellers * scale);
- executeStatement(con, sql.data);
-
- printfPQExpBuffer(&sql,
- "insert into pgbench_accounts(aid,bid,abalance,filler) "
- "select aid, (aid - 1) / %d + 1, 0, '' "
- "from generate_series(1, " INT64_FORMAT ") as aid",
- naccounts, (int64) naccounts * scale);
- executeStatement(con, sql.data);
+ for (int i = 0; i < scale; i += chunk) {
+ executeStatement(con, "begin");
+
+ printfPQExpBuffer(&sql,
+ "insert into pgbench_branches(bid,bbalance) "
+ "select bid + 1, 0 "
+ "from generate_series(%d, %d) as bid", i, i + chunk);
+ //"select bid, 0 "
+ //"from generate_series(1, %d) as bid", nbranches * scale);
+ executeStatement(con, sql.data);
+
+ printfPQExpBuffer(&sql,
+ "insert into pgbench_tellers(tid,bid,tbalance) "
+ "select tid + 1, tid / %d + 1, 0 "
+ "from generate_series(%d, %d) as tid",
+ ntellers, i * ntellers, (i + chunk) * ntellers - 1);
+ //"select tid, (tid - 1) / %d + 1, 0 "
+ //"from generate_series(1, %d) as tid", ntellers, ntellers * scale);
+ executeStatement(con, sql.data);
+
+ printfPQExpBuffer(&sql,
+ "insert into pgbench_accounts(aid,bid,abalance,filler) "
+ "select aid + 1, aid / %d + 1, 0, '' "
+ "from generate_series(" INT64_FORMAT ", " INT64_FORMAT ") as aid",
+ naccounts, (int64) i * naccounts, (int64) (i + chunk) * naccounts - 1);
+ //"select aid, (aid - 1) / %d + 1, 0, '' "
+ //"from generate_series(1, " INT64_FORMAT ") as aid",
+ //naccounts, (int64) naccounts * scale);
+ executeStatement(con, sql.data);
+
+ executeStatement(con, "commit");
+ }
termPQExpBuffer(&sql);
-
- executeStatement(con, "commit");
}
/*
--
2.43.0
From 0eddb156c187d829c4381bc928c5314705928852 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Sun, 9 Nov 2025 20:13:23 +0700
Subject: [PATCH v7 02/18] Getting rid off limit for single transaction size
during data generation
---
src/bin/pgbench/pgbench.c | 15 ++++-----------
1 file changed, 4 insertions(+), 11 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 284a7c860f1..28b72e4cf1f 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -181,12 +181,6 @@ static int64 end_time = 0; /* when to stop in micro seconds, under -T */
*/
static int scale = 1;
-/*
- * scaling factor after which we switch to multiple transactions during
- * data population phase on server side
- */
-static int64 single_txn_scale_limit = 1;
-
/*
* fillfactor. for example, fillfactor = 90 will use only 90 percent
* space during inserts and leave 10 percent free.
@@ -5219,7 +5213,6 @@ static void
initGenerateDataServerSide(PGconn *con)
{
PQExpBufferData sql;
- int chunk = (scale >= single_txn_scale_limit) ? 1 : scale;
fprintf(stderr, "generating data (server-side)...\n");
@@ -5236,13 +5229,13 @@ initGenerateDataServerSide(PGconn *con)
initPQExpBuffer(&sql);
- for (int i = 0; i < scale; i += chunk) {
+ for (int i = 0; i < scale; i++) {
executeStatement(con, "begin");
printfPQExpBuffer(&sql,
"insert into pgbench_branches(bid,bbalance) "
"select bid + 1, 0 "
- "from generate_series(%d, %d) as bid", i, i + chunk);
+ "from generate_series(%d, %d) as bid", i, i + 1);
//"select bid, 0 "
//"from generate_series(1, %d) as bid", nbranches * scale);
executeStatement(con, sql.data);
@@ -5251,7 +5244,7 @@ initGenerateDataServerSide(PGconn *con)
"insert into pgbench_tellers(tid,bid,tbalance) "
"select tid + 1, tid / %d + 1, 0 "
"from generate_series(%d, %d) as tid",
- ntellers, i * ntellers, (i + chunk) * ntellers - 1);
+ ntellers, i * ntellers, (i + 1) * ntellers - 1);
//"select tid, (tid - 1) / %d + 1, 0 "
//"from generate_series(1, %d) as tid", ntellers, ntellers * scale);
executeStatement(con, sql.data);
@@ -5260,7 +5253,7 @@ initGenerateDataServerSide(PGconn *con)
"insert into pgbench_accounts(aid,bid,abalance,filler) "
"select aid + 1, aid / %d + 1, 0, '' "
"from generate_series(" INT64_FORMAT ", " INT64_FORMAT ") as aid",
- naccounts, (int64) i * naccounts, (int64) (i + chunk) * naccounts - 1);
+ naccounts, (int64) i * naccounts, (int64) (i + 1) * naccounts - 1);
//"select aid, (aid - 1) / %d + 1, 0, '' "
//"from generate_series(1, " INT64_FORMAT ") as aid",
//naccounts, (int64) naccounts * scale);
--
2.43.0
From c5659cf474ec273c057668f30a4f435fd02f2da7 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Sun, 9 Nov 2025 20:38:36 +0700
Subject: [PATCH v7 03/18] No need to keep old code in comments
---
src/bin/pgbench/pgbench.c | 7 -------
1 file changed, 7 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 28b72e4cf1f..97895aa9edf 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -5236,8 +5236,6 @@ initGenerateDataServerSide(PGconn *con)
"insert into pgbench_branches(bid,bbalance) "
"select bid + 1, 0 "
"from generate_series(%d, %d) as bid", i, i + 1);
- //"select bid, 0 "
- //"from generate_series(1, %d) as bid", nbranches * scale);
executeStatement(con, sql.data);
printfPQExpBuffer(&sql,
@@ -5245,8 +5243,6 @@ initGenerateDataServerSide(PGconn *con)
"select tid + 1, tid / %d + 1, 0 "
"from generate_series(%d, %d) as tid",
ntellers, i * ntellers, (i + 1) * ntellers - 1);
- //"select tid, (tid - 1) / %d + 1, 0 "
- //"from generate_series(1, %d) as tid", ntellers, ntellers * scale);
executeStatement(con, sql.data);
printfPQExpBuffer(&sql,
@@ -5254,9 +5250,6 @@ initGenerateDataServerSide(PGconn *con)
"select aid + 1, aid / %d + 1, 0, '' "
"from generate_series(" INT64_FORMAT ", " INT64_FORMAT ") as aid",
naccounts, (int64) i * naccounts, (int64) (i + 1) * naccounts - 1);
- //"select aid, (aid - 1) / %d + 1, 0, '' "
- //"from generate_series(1, " INT64_FORMAT ") as aid",
- //naccounts, (int64) naccounts * scale);
executeStatement(con, sql.data);
executeStatement(con, "commit");
--
2.43.0
From e47b52ddf23593dad9375ef5356fd41d0621ede3 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Mon, 10 Nov 2025 19:06:48 +0700
Subject: [PATCH v7 04/18] Adding server-side data generation via unnest
---
src/bin/pgbench/pgbench.c | 199 ++++++++++++++++++++++++++++++++++----
1 file changed, 182 insertions(+), 17 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 97895aa9edf..65d77cdefea 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -161,7 +161,7 @@ typedef struct socket_set
* some configurable parameters */
#define DEFAULT_INIT_STEPS "dtgvp" /* default -I setting */
-#define ALL_INIT_STEPS "dtgGvpf" /* all possible steps */
+#define ALL_INIT_STEPS "dtgGiIvpf" /* all possible steps */
#define LOG_STEP_SECONDS 5 /* seconds between log messages */
#define DEFAULT_NXACTS 10 /* default nxacts */
@@ -171,6 +171,12 @@ typedef struct socket_set
#define MIN_ZIPFIAN_PARAM 1.001 /* minimum parameter for zipfian */
#define MAX_ZIPFIAN_PARAM 1000.0 /* maximum parameter for zipfian */
+/* original single transaction server-side method */
+#define GEN_TYPE_INSERT_ORIGINAL 'G' /* use INSERT .. SELECT generate_series to generate data */
+/* 'one transaction per scale' server-side methods */
+#define GEN_TYPE_INSERT_SERIES 'i' /* use INSERT .. SELECT generate_series to generate data */
+#define GEN_TYPE_INSERT_UNNEST 'I' /* use INSERT .. SELECT unnest to generate data */
+
static int nxacts = 0; /* number of transactions per client */
static int duration = 0; /* duration in seconds */
static int64 end_time = 0; /* when to stop in micro seconds, under -T */
@@ -181,6 +187,11 @@ static int64 end_time = 0; /* when to stop in micro seconds, under -T */
*/
static int scale = 1;
+/*
+ *
+ */
+static char data_generation_type = '?';
+
/*
* fillfactor. for example, fillfactor = 90 will use only 90 percent
* space during inserts and leave 10 percent free.
@@ -914,7 +925,9 @@ usage(void)
" d: drop any existing pgbench tables\n"
" t: create the tables used by the standard pgbench scenario\n"
" g: generate data, client-side\n"
- " G: generate data, server-side\n"
+ " G: generate data, server-side in single transaction\n"
+ " i: server-side (multiple TXNs) INSERT .. SELECT generate_series\n"
+ " I: server-side (multiple TXNs) INSERT .. SELECT unnest\n"
" v: invoke VACUUM on the standard tables\n"
" p: create primary key indexes on the standard tables\n"
" f: create foreign keys between the standard tables\n"
@@ -5203,18 +5216,16 @@ initGenerateDataClientSide(PGconn *con)
}
/*
- * Fill the standard tables with some data generated on the server
- *
- * As already the case with the client-side data generation, the filler
- * column defaults to NULL in pgbench_branches and pgbench_tellers,
- * and is a blank-padded string in pgbench_accounts.
+ * Generating data via INSERT .. SELECT .. FROM generate_series
+ * whole dataset in single transaction
*/
static void
-initGenerateDataServerSide(PGconn *con)
+generateDataInsertSingleTXN(PGconn *con)
{
PQExpBufferData sql;
- fprintf(stderr, "generating data (server-side)...\n");
+ fprintf(stderr, "via INSERT .. SELECT generate_series... in single TXN\n");
+
/*
* we do all of this in one transaction to enable the backend's
@@ -5225,31 +5236,136 @@ initGenerateDataServerSide(PGconn *con)
/* truncate away any old data */
initTruncateTables(con);
+ initPQExpBuffer(&sql);
+
+ printfPQExpBuffer(&sql,
+ "insert into pgbench_branches(bid, bbalance) "
+ "select bid, 0 "
+ "from generate_series(1, %d)", scale * nbranches);
+ executeStatement(con, sql.data);
+
+ printfPQExpBuffer(&sql,
+ "insert into pgbench_tellers(tid, bid, tbalance) "
+ "select tid + 1, tid / %d + 1, 0 "
+ "from generate_series(0, %d) as tid",
+ ntellers, (scale * ntellers) - 1);
+ executeStatement(con, sql.data);
+
+ printfPQExpBuffer(&sql,
+ "insert into pgbench_accounts(aid, bid, abalance, "
+ "filler) "
+ "select aid + 1, aid / %d + 1, 0, '' "
+ "from generate_series(0, " INT64_FORMAT ") as aid",
+ naccounts, (int64) (scale * naccounts) - 1);
+ executeStatement(con, sql.data);
+
executeStatement(con, "commit");
+ termPQExpBuffer(&sql);
+}
+
+
+/*
+ * Generating data via INSERT .. SELECT .. FROM generate_series
+ * One transaction per 'scale'
+ */
+static void
+generateDataInsertSeries(PGconn *con)
+{
+ PQExpBufferData sql;
+
+ fprintf(stderr, "via INSERT .. SELECT generate_series... in multiple TXN(s)\n");
+
initPQExpBuffer(&sql);
- for (int i = 0; i < scale; i++) {
+ executeStatement(con, "begin");
+
+ /* truncate away any old data */
+ initTruncateTables(con);
+
+ executeStatement(con, "commit");
+
+ for (int i = 0; i < scale; i++)
+ {
executeStatement(con, "begin");
printfPQExpBuffer(&sql,
- "insert into pgbench_branches(bid,bbalance) "
- "select bid + 1, 0 "
- "from generate_series(%d, %d) as bid", i, i + 1);
+ "insert into pgbench_branches(bid, bbalance) "
+ "values(%d, 0)", i + 1);
executeStatement(con, sql.data);
printfPQExpBuffer(&sql,
- "insert into pgbench_tellers(tid,bid,tbalance) "
+ "insert into pgbench_tellers(tid, bid, tbalance) "
"select tid + 1, tid / %d + 1, 0 "
"from generate_series(%d, %d) as tid",
ntellers, i * ntellers, (i + 1) * ntellers - 1);
executeStatement(con, sql.data);
printfPQExpBuffer(&sql,
- "insert into pgbench_accounts(aid,bid,abalance,filler) "
+ "insert into pgbench_accounts(aid, bid, abalance, "
+ "filler) "
"select aid + 1, aid / %d + 1, 0, '' "
- "from generate_series(" INT64_FORMAT ", " INT64_FORMAT ") as aid",
- naccounts, (int64) i * naccounts, (int64) (i + 1) * naccounts - 1);
+ "from generate_series(" INT64_FORMAT ", "
+ INT64_FORMAT ") as aid",
+ naccounts, (int64) i * naccounts,
+ (int64) (i + 1) * naccounts - 1);
+ executeStatement(con, sql.data);
+
+ executeStatement(con, "commit");
+ }
+
+ termPQExpBuffer(&sql);
+}
+
+/*
+ * Generating data via INSERT .. SELECT .. FROM unnest
+ * One transaction per 'scale'
+ */
+static void
+generateDataInsertUnnest(PGconn *con)
+{
+ PQExpBufferData sql;
+
+ fprintf(stderr, "via INSERT .. SELECT unnest...\n");
+
+ initPQExpBuffer(&sql);
+
+ executeStatement(con, "begin");
+
+ /* truncate away any old data */
+ initTruncateTables(con);
+
+ executeStatement(con, "commit");
+
+ for (int s = 0; s < scale; s++)
+ {
+ executeStatement(con, "begin");
+
+ printfPQExpBuffer(&sql,
+ "insert into pgbench_branches(bid,bbalance) "
+ "values(%d, 0)", s + 1);
+ executeStatement(con, sql.data);
+
+ printfPQExpBuffer(&sql,
+ "insert into pgbench_tellers(tid, bid, tbalance) "
+ "select unnest(array_agg(s.i order by s.i)) as tid, "
+ "%d as bid, 0 as tbalance "
+ "from generate_series(%d, %d) as s(i)",
+ s + 1, s * ntellers + 1, (s + 1) * ntellers);
+ executeStatement(con, sql.data);
+
+ printfPQExpBuffer(&sql,
+ "with data as ("
+ " select generate_series(" INT64_FORMAT ", "
+ INT64_FORMAT ") as i) "
+ "insert into pgbench_accounts(aid, bid, "
+ "abalance, filler) "
+ "select unnest(aid), unnest(bid), 0 as abalance, "
+ "'' as filler "
+ "from (select array_agg(i+1) aid, "
+ "array_agg(i/%d + 1) bid from data)",
+ (int64) s * naccounts + 1,
+ (int64) (s + 1) * naccounts, naccounts);
executeStatement(con, sql.data);
executeStatement(con, "commit");
@@ -5258,6 +5374,32 @@ initGenerateDataServerSide(PGconn *con)
termPQExpBuffer(&sql);
}
+/*
+ * Fill the standard tables with some data generated on the server
+ *
+ * As already the case with the client-side data generation, the filler
+ * column defaults to NULL in pgbench_branches and pgbench_tellers,
+ * and is a blank-padded string in pgbench_accounts.
+ */
+static void
+initGenerateDataServerSide(PGconn *con)
+{
+ fprintf(stderr, "generating data (server-side) ");
+
+ switch (data_generation_type)
+ {
+ case GEN_TYPE_INSERT_ORIGINAL:
+ generateDataInsertSingleTXN(con);
+ break;
+ case GEN_TYPE_INSERT_SERIES:
+ generateDataInsertSeries(con);
+ break;
+ case GEN_TYPE_INSERT_UNNEST:
+ generateDataInsertUnnest(con);
+ break;
+ }
+}
+
/*
* Invoke vacuum on the standard tables
*/
@@ -5341,6 +5483,8 @@ initCreateFKeys(PGconn *con)
static void
checkInitSteps(const char *initialize_steps)
{
+ char data_init_type = 0;
+
if (initialize_steps[0] == '\0')
pg_fatal("no initialization steps specified");
@@ -5352,7 +5496,26 @@ checkInitSteps(const char *initialize_steps)
pg_log_error_detail("Allowed step characters are: \"" ALL_INIT_STEPS "\".");
exit(1);
}
+
+ switch (*step)
+ {
+ case 'G':
+ data_init_type++;
+ data_generation_type = *step;
+ break;
+ case 'i':
+ data_init_type++;
+ data_generation_type = *step;
+ break;
+ case 'I':
+ data_init_type++;
+ data_generation_type = *step;
+ break;
+ }
}
+
+ if (data_init_type > 1)
+ pg_log_error("WARNING! More than one type of server-side data generation is requested");
}
/*
@@ -5395,6 +5558,8 @@ runInitSteps(const char *initialize_steps)
initGenerateDataClientSide(con);
break;
case 'G':
+ case 'i':
+ case 'I':
op = "server-side generate";
initGenerateDataServerSide(con);
break;
--
2.43.0
From 5e1827b889b283f50299ce6ab1a73f9f55a4a84f Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Mon, 10 Nov 2025 20:00:56 +0700
Subject: [PATCH v7 05/18] Fixing typo in query
---
src/bin/pgbench/pgbench.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 65d77cdefea..03e37df4434 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -5241,7 +5241,8 @@ generateDataInsertSingleTXN(PGconn *con)
printfPQExpBuffer(&sql,
"insert into pgbench_branches(bid, bbalance) "
"select bid, 0 "
- "from generate_series(1, %d)", scale * nbranches);
+ "from generate_series(1, %d) as bid",
+ scale * nbranches);
executeStatement(con, sql.data);
printfPQExpBuffer(&sql,
--
2.43.0
From 7ca86521fda6929b8e0de3fc77dcbb8984009c88 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Tue, 11 Nov 2025 19:39:45 +0700
Subject: [PATCH v7 06/18] Adding support for COPY BINARY mode
---
src/bin/pgbench/pgbench.c | 393 ++++++++++++++++++++++++++++++++++++--
1 file changed, 381 insertions(+), 12 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 03e37df4434..71aa1d9479f 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -161,7 +161,7 @@ typedef struct socket_set
* some configurable parameters */
#define DEFAULT_INIT_STEPS "dtgvp" /* default -I setting */
-#define ALL_INIT_STEPS "dtgGiIvpf" /* all possible steps */
+#define ALL_INIT_STEPS "dtgCGiIvpf" /* all possible steps */
#define LOG_STEP_SECONDS 5 /* seconds between log messages */
#define DEFAULT_NXACTS 10 /* default nxacts */
@@ -176,6 +176,8 @@ typedef struct socket_set
/* 'one transaction per scale' server-side methods */
#define GEN_TYPE_INSERT_SERIES 'i' /* use INSERT .. SELECT generate_series to generate data */
#define GEN_TYPE_INSERT_UNNEST 'I' /* use INSERT .. SELECT unnest to generate data */
+#define GEN_TYPE_COPY_ORIGINAL 'g' /* use COPY .. FROM STDIN .. TEXT to generate data */
+#define GEN_TYPE_COPY_BINARY 'C' /* use COPY .. FROM STDIN .. BINARY to generate data */
static int nxacts = 0; /* number of transactions per client */
static int duration = 0; /* duration in seconds */
@@ -188,10 +190,17 @@ static int64 end_time = 0; /* when to stop in micro seconds, under -T */
static int scale = 1;
/*
- *
+ * mode of data generation to use
*/
static char data_generation_type = '?';
+/*
+ * COPY FROM BINARY execution buffer
+ */
+#define BIN_COPY_BUF_SIZE 102400 /* maximum buffer size for COPY FROM BINARY */
+static char *bin_copy_buffer = NULL; /* buffer for COPY FROM BINARY */
+static int32_t bin_copy_buffer_length = 0; /* current buffer size */
+
/*
* fillfactor. for example, fillfactor = 90 will use only 90 percent
* space during inserts and leave 10 percent free.
@@ -861,7 +870,8 @@ static int wait_on_socket_set(socket_set *sa, int64 usecs);
static bool socket_has_input(socket_set *sa, int fd, int idx);
/* callback used to build rows for COPY during data loading */
-typedef void (*initRowMethod) (PQExpBufferData *sql, int64 curr);
+typedef void (*initRowMethod) (PQExpBufferData *sql, int64 curr);
+typedef void (*initRowMethodBin) (PGconn *con, PGresult *res, int64_t curr, int32_t parent);
/* callback functions for our flex lexer */
static const PsqlScanCallbacks pgbench_callbacks = {
@@ -925,6 +935,7 @@ usage(void)
" d: drop any existing pgbench tables\n"
" t: create the tables used by the standard pgbench scenario\n"
" g: generate data, client-side\n"
+ " C: client-side (single TNX) COPY .. FROM STDIN .. BINARY\n"
" G: generate data, server-side in single transaction\n"
" i: server-side (multiple TXNs) INSERT .. SELECT generate_series\n"
" I: server-side (multiple TXNs) INSERT .. SELECT unnest\n"
@@ -5191,9 +5202,9 @@ initPopulateTable(PGconn *con, const char *table, int64 base,
* a blank-padded string in pgbench_accounts.
*/
static void
-initGenerateDataClientSide(PGconn *con)
+initGenerateDataClientSideText(PGconn *con)
{
- fprintf(stderr, "generating data (client-side)...\n");
+ fprintf(stderr, "TEXT mode...\n");
/*
* we do all of this in one transaction to enable the backend's
@@ -5209,12 +5220,373 @@ initGenerateDataClientSide(PGconn *con)
* already exist
*/
initPopulateTable(con, "pgbench_branches", nbranches, initBranch);
- initPopulateTable(con, "pgbench_tellers", ntellers, initTeller);
+ initPopulateTable(con, "pgbench_tellers", ntellers, initTeller);
initPopulateTable(con, "pgbench_accounts", naccounts, initAccount);
executeStatement(con, "commit");
}
+
+/*
+ * Dumps binary buffer to file (purely for debugging)
+ */
+static void
+dumpBufferToFile(char *filename)
+{
+ FILE *file_ptr;
+ size_t bytes_written;
+
+ file_ptr = fopen(filename, "wb");
+ if (file_ptr == NULL)
+ {
+ fprintf(stderr, "Error opening file %s\n", filename);
+ return; // EXIT_FAILURE;
+ }
+
+ bytes_written = fwrite(bin_copy_buffer, 1, bin_copy_buffer_length, file_ptr);
+
+ if (bytes_written != bin_copy_buffer_length)
+ {
+ fprintf(stderr, "Error writing to file or incomplete write\n");
+ fclose(file_ptr);
+ return; // EXIT_FAILURE;
+ }
+
+ fclose(file_ptr);
+}
+
+/*
+ * Save char data to buffer
+ */
+static void
+bufferCharData(char *src, int32_t len)
+{
+ memcpy((char *) bin_copy_buffer + bin_copy_buffer_length, (char *) src, len);
+ bin_copy_buffer_length += len;
+}
+
+/*
+ * Converts platform byte order into network byte order
+ * SPARC doesn't reqire that
+ */
+static void
+bufferData(void *src, int32_t len)
+{
+#ifdef __sparc__
+ bufferCharData(src, len);
+#else
+ if (len == 1)
+ bufferCharData(src, len);
+ else
+ for (int32_t i = 0; i < len; i++)
+ {
+ ((char *) bin_copy_buffer + bin_copy_buffer_length)[i] =
+ ((char *) src)[len - i - 1];
+ }
+
+ bin_copy_buffer_length += len;
+#endif
+}
+
+/*
+ * adds column counter
+ */
+static void
+addColumnCounter(int16_t n)
+{
+ bufferData((void *) &n, sizeof(n));
+}
+
+/*
+ * adds column with NULL value
+ */
+static void
+addNullColumn()
+{
+ int32_t null = -1;
+ bufferData((void *) &null, sizeof(null));
+}
+
+/*
+ * adds column with int8 value
+ */
+static void
+addInt8Column(int8_t value)
+{
+ int8_t data = value;
+ int32_t size = sizeof(data);
+ bufferData((void *) &size, sizeof(size));
+ bufferData((void *) &data, sizeof(data));
+}
+
+/*
+ * adds column with int16 value
+ */
+static void
+addInt16Column(int16_t value)
+{
+ int16_t data = value;
+ int32_t size = sizeof(data);
+ bufferData((void *) &size, sizeof(size));
+ bufferData((void *) &data, sizeof(data));
+}
+
+/*
+ * adds column with inti32 value
+ */
+static void
+addInt32Column(int32_t value)
+{
+ int32_t data = value;
+ int32_t size = sizeof(data);
+ bufferData((void *) &size, sizeof(size));
+ bufferData((void *) &data, sizeof(data));
+}
+
+/*
+ * adds column with inti64 value
+ */
+static void
+addInt64Column(int64_t value)
+{
+ int64_t data = value;
+ int32_t size = sizeof(data);
+ bufferData((void *) &size, sizeof(size));
+ bufferData((void *) &data, sizeof(data));
+}
+
+/*
+ * adds column with char value
+ */
+static void
+addCharColumn(char *value)
+{
+ int32_t size = strlen(value);
+ bufferData((void *) &size, sizeof(size));
+ bufferCharData(value, size);
+}
+
+/*
+ * Starts communication with server for COPY FROM BINARY statement
+ */
+static void
+sendBinaryCopyHeader(PGconn *con)
+{
+ char header[] = {'P','G','C','O','P','Y','\n','\377','\r','\n','\0',
+ '\0','\0','\0','\0',
+ '\0','\0','\0','\0' };
+
+ PQputCopyData(con, header, 19);
+}
+
+/*
+ * Finishes communication with server for COPY FROM BINARY statement
+ */
+static void
+sendBinaryCopyTrailer(PGconn *con)
+{
+ static char trailer[] = { 0xFF, 0xFF };
+
+ PQputCopyData(con, trailer, 2);
+}
+
+/*
+ * Flashes current buffer over network if needed
+ */
+static void
+flushBuffer(PGconn *con, PGresult *res, int16_t row_len)
+{
+ if (bin_copy_buffer_length + row_len > BIN_COPY_BUF_SIZE)
+ {
+ /* flush current buffer */
+ if (PQresultStatus(res) == PGRES_COPY_IN)
+ PQputCopyData(con, (char *) bin_copy_buffer, bin_copy_buffer_length);
+ bin_copy_buffer_length = 0;
+ }
+}
+
+/*
+ * Sends current branch row to buffer
+ */
+static void
+initBranchBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
+{
+ /*
+ * Each row has following extra bytes:
+ * - 2 bytes for number of columns
+ * - 4 bytes as length for each column
+ */
+ int16_t max_row_len = 35 + 2 + 4*3; /* max row size is 32 */
+
+ flushBuffer(con, res, max_row_len);
+
+ addColumnCounter(2);
+
+ addInt32Column(curr + 1);
+ addInt32Column(0);
+}
+
+/*
+ * Sends current teller row to buffer
+ */
+static void
+initTellerBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
+{
+ /*
+ * Each row has following extra bytes:
+ * - 2 bytes for number of columns
+ * - 4 bytes as length for each column
+ */
+ int16_t max_row_len = 40 + 2 + 4*4; /* max row size is 40 */
+
+ flushBuffer(con, res, max_row_len);
+
+ addColumnCounter(3);
+
+ addInt32Column(curr + 1);
+ addInt32Column(curr / parent + 1);
+ addInt32Column(0);
+}
+
+/*
+ * Sends current account row to buffer
+ */
+static void
+initAccountBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
+{
+ /*
+ * Each row has following extra bytes:
+ * - 2 bytes for number of columns
+ * - 4 bytes as length for each column
+ */
+ int16_t max_row_len = 250 + 2 + 4*4; /* max row size is 250 for int64 */
+
+ flushBuffer(con, res, max_row_len);
+
+ addColumnCounter(3);
+
+ if (scale <= SCALE_32BIT_THRESHOLD)
+ addInt32Column(curr + 1);
+ else
+ addInt64Column(curr);
+
+ addInt32Column(curr / parent + 1);
+ addInt32Column(0);
+}
+
+/*
+ * Universal wrapper for sending data in binary format
+ */
+static void
+initPopulateTableBinary(PGconn *con, char *table, char *columns,
+ int64_t base, initRowMethodBin init_row)
+{
+ int n;
+ PGresult *res;
+ char copy_statement[256];
+ const char *copy_statement_fmt = "copy %s (%s) from stdin (format binary)";
+ int64_t total = base * scale;
+
+ bin_copy_buffer_length = 0;
+
+ /* Use COPY with FREEZE on v14 and later for all ordinary tables */
+ if ((PQserverVersion(con) >= 140000) &&
+ get_table_relkind(con, table) == RELKIND_RELATION)
+ copy_statement_fmt = "copy %s (%s) from stdin with (format binary, freeze on)";
+
+ n = pg_snprintf(copy_statement, sizeof(copy_statement), copy_statement_fmt, table, columns);
+ if (n >= sizeof(copy_statement))
+ pg_fatal("invalid buffer size: must be at least %d characters long", n);
+ else if (n == -1)
+ pg_fatal("invalid format string");
+
+ res = PQexec(con, copy_statement);
+
+ if (PQresultStatus(res) != PGRES_COPY_IN)
+ pg_fatal("unexpected copy in result: %s", PQerrorMessage(con));
+ PQclear(res);
+
+
+ sendBinaryCopyHeader(con);
+
+ for (int64_t i = 0; i < total; i++)
+ {
+ init_row(con, res, i, base);
+ }
+
+ if (PQresultStatus(res) == PGRES_COPY_IN)
+ PQputCopyData(con, (char *) bin_copy_buffer, bin_copy_buffer_length);
+ else
+ fprintf(stderr, "Unexpected mode %d instead of %d\n", PQresultStatus(res), PGRES_COPY_IN);
+
+ sendBinaryCopyTrailer(con);
+
+ if (PQresultStatus(res) == PGRES_COPY_IN)
+ {
+ if (PQputCopyEnd(con, NULL) == 1) /* success */
+ {
+ res = PQgetResult(con);
+ if (PQresultStatus(res) != PGRES_COMMAND_OK)
+ fprintf(stderr, "Error: %s\n", PQerrorMessage(con));
+ PQclear(res);
+ }
+ else
+ fprintf(stderr, "Error: %s\n", PQerrorMessage(con));
+ }
+}
+
+/*
+ * Wrapper for binary data load
+ */
+static void
+initGenerateDataClientSideBinary(PGconn *con)
+{
+
+ fprintf(stderr, "BINARY mode...\n");
+
+ bin_copy_buffer = pg_malloc(BIN_COPY_BUF_SIZE);
+ bin_copy_buffer_length = 0;
+
+ /*
+ * we do all of this in one transaction to enable the backend's
+ * data-loading optimizations
+ */
+ executeStatement(con, "begin");
+
+ /* truncate away any old data */
+ initTruncateTables(con);
+
+ initPopulateTableBinary(con, "pgbench_branches", "bid, bbalance",
+ nbranches, initBranchBinary);
+ initPopulateTableBinary(con, "pgbench_tellers", "tid, bid, tbalance",
+ ntellers, initTellerBinary);
+ initPopulateTableBinary(con, "pgbench_accounts", "aid, bid, abalance",
+ naccounts, initAccountBinary);
+
+ executeStatement(con, "commit");
+
+ pg_free(bin_copy_buffer);
+}
+
+/*
+ * Fill the standard tables with some data generated and sent from the client.
+ */
+static void
+initGenerateDataClientSide(PGconn *con)
+{
+ fprintf(stderr, "generating data (client-side) in ");
+
+ switch (data_generation_type)
+ {
+ case GEN_TYPE_COPY_ORIGINAL:
+ initGenerateDataClientSideText(con);
+ break;
+ case GEN_TYPE_COPY_BINARY:
+ initGenerateDataClientSideBinary(con);
+ break;
+ }
+}
+
/*
* Generating data via INSERT .. SELECT .. FROM generate_series
* whole dataset in single transaction
@@ -5500,14 +5872,10 @@ checkInitSteps(const char *initialize_steps)
switch (*step)
{
+ case 'g':
+ case 'C':
case 'G':
- data_init_type++;
- data_generation_type = *step;
- break;
case 'i':
- data_init_type++;
- data_generation_type = *step;
- break;
case 'I':
data_init_type++;
data_generation_type = *step;
@@ -5555,6 +5923,7 @@ runInitSteps(const char *initialize_steps)
initCreateTables(con);
break;
case 'g':
+ case 'C':
op = "client-side generate";
initGenerateDataClientSide(con);
break;
--
2.43.0
From 4aa0ac05765edf6b5f0c13e18ac677287ce78206 Mon Sep 17 00:00:00 2001
From: Fujii Masao <[email protected]>
Date: Fri, 14 Nov 2025 22:40:39 +0900
Subject: [PATCH v7 07/18] pgbench: Fix assertion failure with multiple
\syncpipeline in pipeline mode.
Previously, when pgbench ran a custom script that triggered retriable errors
(e.g., deadlocks) followed by multiple \syncpipeline commands in pipeline mode,
the following assertion failure could occur:
Assertion failed: (res == ((void*)0)), function discardUntilSync, file pgbench.c, line 3594.
The issue was that discardUntilSync() assumed a pipeline sync result
(PGRES_PIPELINE_SYNC) would always be followed by either another sync result
or NULL. This assumption was incorrect: when multiple sync requests were sent,
a sync result could instead be followed by another result type. In such cases,
discardUntilSync() mishandled the results, leading to the assertion failure.
This commit fixes the issue by making discardUntilSync() correctly handle cases
where a pipeline sync result is followed by other result types. It now continues
discarding results until another pipeline sync followed by NULL is reached.
Backpatched to v17, where support for \syncpipeline command in pgbench was
introduced.
Author: Yugo Nagata <[email protected]>
Reviewed-by: Chao Li <[email protected]>
Reviewed-by: Fujii Masao <[email protected]>
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 17
---
src/bin/pgbench/pgbench.c | 39 ++++++++++++++++++++++++++++-----------
1 file changed, 28 insertions(+), 11 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index d8764ba6fe0..a425176ecdc 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -3563,14 +3563,18 @@ doRetry(CState *st, pg_time_usec_t *now)
}
/*
- * Read results and discard it until a sync point.
+ * Read and discard results until the last sync point.
*/
static int
discardUntilSync(CState *st)
{
bool received_sync = false;
- /* send a sync */
+ /*
+ * Send a Sync message to ensure at least one PGRES_PIPELINE_SYNC is
+ * received and to avoid an infinite loop, since all earlier ones may have
+ * already been received.
+ */
if (!PQpipelineSync(st->con))
{
pg_log_error("client %d aborted: failed to send a pipeline sync",
@@ -3578,29 +3582,42 @@ discardUntilSync(CState *st)
return 0;
}
- /* receive PGRES_PIPELINE_SYNC and null following it */
+ /*
+ * Continue reading results until the last sync point, i.e., until
+ * reaching null just after PGRES_PIPELINE_SYNC.
+ */
for (;;)
{
PGresult *res = PQgetResult(st->con);
+ if (PQstatus(st->con) == CONNECTION_BAD)
+ {
+ pg_log_error("client %d aborted while rolling back the transaction after an error; perhaps the backend died while processing",
+ st->id);
+ PQclear(res);
+ return 0;
+ }
+
if (PQresultStatus(res) == PGRES_PIPELINE_SYNC)
received_sync = true;
- else if (received_sync)
+ else if (received_sync && res == NULL)
{
- /*
- * PGRES_PIPELINE_SYNC must be followed by another
- * PGRES_PIPELINE_SYNC or NULL; otherwise, assert failure.
- */
- Assert(res == NULL);
-
/*
* Reset ongoing sync count to 0 since all PGRES_PIPELINE_SYNC
* results have been discarded.
*/
st->num_syncs = 0;
- PQclear(res);
break;
}
+ else
+ {
+ /*
+ * If a PGRES_PIPELINE_SYNC is followed by something other than
+ * PGRES_PIPELINE_SYNC or NULL, another PGRES_PIPELINE_SYNC will
+ * appear later. Reset received_sync to false to wait for it.
+ */
+ received_sync = false;
+ }
PQclear(res);
}
--
2.43.0
From 9c4f19055597e9adb25e65c2aa8bedf20a09e13d Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Fri, 21 Nov 2025 19:05:58 +0700
Subject: [PATCH v7 08/18] Setting empty string as default value in filler
column
---
src/bin/pgbench/pgbench.c | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 967f6ce6984..03b5e5c28f0 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -4985,26 +4985,26 @@ initCreateTables(PGconn *con)
static const struct ddlinfo DDLs[] = {
{
"pgbench_history",
- "tid int,bid int,aid int,delta int,mtime timestamp,filler char(22)",
- "tid int,bid int,aid bigint,delta int,mtime timestamp,filler char(22)",
+ "tid int,bid int,aid int,delta int,mtime timestamp,filler char(22) default ''",
+ "tid int,bid int,aid bigint,delta int,mtime timestamp,filler char(22) default ''",
0
},
{
"pgbench_tellers",
- "tid int not null,bid int,tbalance int,filler char(84)",
- "tid int not null,bid int,tbalance int,filler char(84)",
+ "tid int not null,bid int,tbalance int,filler char(84) default ''",
+ "tid int not null,bid int,tbalance int,filler char(84) default ''",
1
},
{
"pgbench_accounts",
- "aid int not null,bid int,abalance int,filler char(84)",
- "aid bigint not null,bid int,abalance int,filler char(84)",
+ "aid int not null,bid int,abalance int,filler char(84) default ''",
+ "aid bigint not null,bid int,abalance int,filler char(84) default ''",
1
},
{
"pgbench_branches",
- "bid int not null,bbalance int,filler char(88)",
- "bid int not null,bbalance int,filler char(88)",
+ "bid int not null,bbalance int,filler char(88) default ''",
+ "bid int not null,bbalance int,filler char(88) default ''",
1
}
};
--
2.43.0
From 2aabaa52dffdb78fbefaef95163881c15e18ef29 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <[email protected]>
Date: Fri, 21 Nov 2025 15:03:11 +0200
Subject: [PATCH v7 09/18] Use strtoi64() in pgbench, replacing its open-coded
implementation
Makes the code a little simpler.
The old implementation accepted trailing whitespace, but that was
unnecessary. Firstly, its sibling function for parsing decimals,
strtodouble(), does not accept trailing whitespace. Secondly, none of
the callers can pass a string with trailing whitespace to it.
In the passing, check specifically for ERANGE before printing the "out
of range" error. On some systems, strtoul() and strtod() return EINVAL
on an empty or all-spaces string, and "invalid input syntax" is more
appropriate for that than "out of range". For the existing
strtodouble() function this is purely academical because it's never
called with errorOK==false, but let's be tidy. (Perhaps we should
remove the dead codepaths altogether, but I'll leave that for another
day.)
Reviewed-by: Chao Li <[email protected]>
Reviewed-by: Yuefei Shi <[email protected]>
Reviewed-by: Neil Chen <[email protected]>
Discussion: https://www.postgresql.org/message-id/[email protected]
---
src/bin/pgbench/pgbench.c | 83 +++++++++------------------------------
1 file changed, 19 insertions(+), 64 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index a425176ecdc..68774a59efd 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -982,13 +982,17 @@ usage(void)
progname, progname, PACKAGE_BUGREPORT, PACKAGE_NAME, PACKAGE_URL);
}
-/* return whether str matches "^\s*[-+]?[0-9]+$" */
+/*
+ * Return whether str matches "^\s*[-+]?[0-9]+$"
+ *
+ * This should agree with strtoint64() on what's accepted, ignoring overflows.
+ */
static bool
is_an_int(const char *str)
{
const char *ptr = str;
- /* skip leading spaces; cast is consistent with strtoint64 */
+ /* skip leading spaces */
while (*ptr && isspace((unsigned char) *ptr))
ptr++;
@@ -1012,9 +1016,6 @@ is_an_int(const char *str)
/*
* strtoint64 -- convert a string to 64-bit integer
*
- * This function is a slightly modified version of pg_strtoint64() from
- * src/backend/utils/adt/numutils.c.
- *
* The function returns whether the conversion worked, and if so
* "*result" is set to the result.
*
@@ -1023,71 +1024,25 @@ is_an_int(const char *str)
bool
strtoint64(const char *str, bool errorOK, int64 *result)
{
- const char *ptr = str;
- int64 tmp = 0;
- bool neg = false;
-
- /*
- * Do our own scan, rather than relying on sscanf which might be broken
- * for long long.
- *
- * As INT64_MIN can't be stored as a positive 64 bit integer, accumulate
- * value as a negative number.
- */
-
- /* skip leading spaces */
- while (*ptr && isspace((unsigned char) *ptr))
- ptr++;
-
- /* handle sign */
- if (*ptr == '-')
- {
- ptr++;
- neg = true;
- }
- else if (*ptr == '+')
- ptr++;
+ char *end;
- /* require at least one digit */
- if (unlikely(!isdigit((unsigned char) *ptr)))
- goto invalid_syntax;
+ errno = 0;
+ *result = strtoi64(str, &end, 10);
- /* process digits */
- while (*ptr && isdigit((unsigned char) *ptr))
+ if (unlikely(errno == ERANGE))
{
- int8 digit = (*ptr++ - '0');
-
- if (unlikely(pg_mul_s64_overflow(tmp, 10, &tmp)) ||
- unlikely(pg_sub_s64_overflow(tmp, digit, &tmp)))
- goto out_of_range;
+ if (!errorOK)
+ pg_log_error("value \"%s\" is out of range for type bigint", str);
+ return false;
}
- /* allow trailing whitespace, but not other trailing chars */
- while (*ptr != '\0' && isspace((unsigned char) *ptr))
- ptr++;
-
- if (unlikely(*ptr != '\0'))
- goto invalid_syntax;
-
- if (!neg)
+ if (unlikely(errno != 0 || end == str || *end != '\0'))
{
- if (unlikely(tmp == PG_INT64_MIN))
- goto out_of_range;
- tmp = -tmp;
+ if (!errorOK)
+ pg_log_error("invalid input syntax for type bigint: \"%s\"", str);
+ return false;
}
-
- *result = tmp;
return true;
-
-out_of_range:
- if (!errorOK)
- pg_log_error("value \"%s\" is out of range for type bigint", str);
- return false;
-
-invalid_syntax:
- if (!errorOK)
- pg_log_error("invalid input syntax for type bigint: \"%s\"", str);
- return false;
}
/* convert string to double, detecting overflows/underflows */
@@ -1099,14 +1054,14 @@ strtodouble(const char *str, bool errorOK, double *dv)
errno = 0;
*dv = strtod(str, &end);
- if (unlikely(errno != 0))
+ if (unlikely(errno == ERANGE))
{
if (!errorOK)
pg_log_error("value \"%s\" is out of range for type double", str);
return false;
}
- if (unlikely(end == str || *end != '\0'))
+ if (unlikely(errno != 0 || end == str || *end != '\0'))
{
if (!errorOK)
pg_log_error("invalid input syntax for type double: \"%s\"", str);
--
2.43.0
From dcb85d26f8132eaaf9d096e814b9bda49db7d478 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Fri, 21 Nov 2025 20:06:24 +0700
Subject: [PATCH v7 10/18] Switching COPY FROM BINARY ti run in multiple
transactions
---
src/bin/pgbench/pgbench.c | 27 ++++++++++++++++-----------
1 file changed, 16 insertions(+), 11 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 03b5e5c28f0..6b89007a63b 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -5496,20 +5496,20 @@ initAccountBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
*/
static void
initPopulateTableBinary(PGconn *con, char *table, char *columns,
- int64_t base, initRowMethodBin init_row)
+ int counter, int64_t base, initRowMethodBin init_row)
{
int n;
PGresult *res;
char copy_statement[256];
const char *copy_statement_fmt = "copy %s (%s) from stdin (format binary)";
- int64_t total = base * scale;
+ int64_t start = base * counter;
bin_copy_buffer_length = 0;
/* Use COPY with FREEZE on v14 and later for all ordinary tables */
if ((PQserverVersion(con) >= 140000) &&
get_table_relkind(con, table) == RELKIND_RELATION)
- copy_statement_fmt = "copy %s (%s) from stdin with (format binary, freeze on)";
+ copy_statement_fmt = "copy %s (%s) from stdin with (format binary)";
n = pg_snprintf(copy_statement, sizeof(copy_statement), copy_statement_fmt, table, columns);
if (n >= sizeof(copy_statement))
@@ -5526,7 +5526,7 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
sendBinaryCopyHeader(con);
- for (int64_t i = 0; i < total; i++)
+ for (int64_t i = start; i < start + base; i++)
{
init_row(con, res, i, base);
}
@@ -5573,15 +5573,20 @@ initGenerateDataClientSideBinary(PGconn *con)
/* truncate away any old data */
initTruncateTables(con);
- initPopulateTableBinary(con, "pgbench_branches", "bid, bbalance",
- nbranches, initBranchBinary);
- initPopulateTableBinary(con, "pgbench_tellers", "tid, bid, tbalance",
- ntellers, initTellerBinary);
- initPopulateTableBinary(con, "pgbench_accounts", "aid, bid, abalance",
- naccounts, initAccountBinary);
-
executeStatement(con, "commit");
+ for (int i = 0; i < scale; i++)
+ {
+ initPopulateTableBinary(con, "pgbench_branches", "bid, bbalance",
+ i, nbranches, initBranchBinary);
+ initPopulateTableBinary(con, "pgbench_tellers", "tid, bid, tbalance",
+ i, ntellers, initTellerBinary);
+ initPopulateTableBinary(con, "pgbench_accounts", "aid, bid, abalance",
+ i, naccounts, initAccountBinary);
+
+ executeStatement(con, "commit");
+ }
+
pg_free(bin_copy_buffer);
}
--
2.43.0
From b8e28881225234fd00b55235bc60fad2dc60b544 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Sat, 22 Nov 2025 17:06:00 +0700
Subject: [PATCH v7 11/18] Adding tests for new modes of data generation
---
src/bin/pgbench/pgbench.c | 21 ++++----
src/bin/pgbench/t/001_pgbench_with_server.pl | 52 +++++++++++++++++---
2 files changed, 56 insertions(+), 17 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 6b89007a63b..dd4e5d5e056 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -164,7 +164,7 @@ typedef struct socket_set
#define ALL_INIT_STEPS "dtgCGiIvpf" /* all possible steps */
#define LOG_STEP_SECONDS 5 /* seconds between log messages */
-#define DEFAULT_NXACTS 10 /* default nxacts */
+#define DEFAULT_NXACTS 10 /* default nxacts */
#define MIN_GAUSSIAN_PARAM 2.0 /* minimum parameter for gauss */
@@ -192,7 +192,7 @@ static int scale = 1;
/*
* mode of data generation to use
*/
-static char data_generation_type = '?';
+static char data_generation_type = GEN_TYPE_COPY_ORIGINAL;
/*
* COPY FROM BINARY execution buffer
@@ -4985,26 +4985,26 @@ initCreateTables(PGconn *con)
static const struct ddlinfo DDLs[] = {
{
"pgbench_history",
- "tid int,bid int,aid int,delta int,mtime timestamp,filler char(22) default ''",
- "tid int,bid int,aid bigint,delta int,mtime timestamp,filler char(22) default ''",
+ "tid int,bid int,aid int,delta int,mtime timestamp,filler char(22) default '?'",
+ "tid int,bid int,aid bigint,delta int,mtime timestamp,filler char(22) default '?'",
0
},
{
"pgbench_tellers",
- "tid int not null,bid int,tbalance int,filler char(84) default ''",
- "tid int not null,bid int,tbalance int,filler char(84) default ''",
+ "tid int not null,bid int,tbalance int,filler char(84)",
+ "tid int not null,bid int,tbalance int,filler char(84)",
1
},
{
"pgbench_accounts",
- "aid int not null,bid int,abalance int,filler char(84) default ''",
- "aid bigint not null,bid int,abalance int,filler char(84) default ''",
+ "aid int not null,bid int,abalance int,filler char(84) default '?'",
+ "aid bigint not null,bid int,abalance int,filler char(84) default '?'",
1
},
{
"pgbench_branches",
- "bid int not null,bbalance int,filler char(88) default ''",
- "bid int not null,bbalance int,filler char(88) default ''",
+ "bid int not null,bbalance int,filler char(88)",
+ "bid int not null,bbalance int,filler char(88)",
1
}
};
@@ -7837,6 +7837,7 @@ main(int argc, char **argv)
}
}
+ checkInitSteps(initialize_steps);
runInitSteps(initialize_steps);
exit(0);
}
diff --git a/src/bin/pgbench/t/001_pgbench_with_server.pl b/src/bin/pgbench/t/001_pgbench_with_server.pl
index 581e9af7907..a377048ead1 100644
--- a/src/bin/pgbench/t/001_pgbench_with_server.pl
+++ b/src/bin/pgbench/t/001_pgbench_with_server.pl
@@ -16,25 +16,30 @@ sub check_data_state
local $Test::Builder::Level = $Test::Builder::Level + 1;
my $node = shift;
my $type = shift;
+ my $sql_result;
- my $sql_result = $node->safe_psql('postgres',
- 'SELECT count(*) AS null_count FROM pgbench_accounts WHERE filler IS NULL LIMIT 10;'
- );
- is($sql_result, '0',
- "$type: filler column of pgbench_accounts has no NULL data");
$sql_result = $node->safe_psql('postgres',
'SELECT count(*) AS null_count FROM pgbench_branches WHERE filler IS NULL;'
);
is($sql_result, '1',
"$type: filler column of pgbench_branches has only NULL data");
+
$sql_result = $node->safe_psql('postgres',
'SELECT count(*) AS null_count FROM pgbench_tellers WHERE filler IS NULL;'
);
is($sql_result, '10',
"$type: filler column of pgbench_tellers has only NULL data");
+
+ $sql_result = $node->safe_psql('postgres',
+ 'SELECT count(*) AS null_count FROM pgbench_accounts WHERE filler IS NULL LIMIT 10;'
+ );
+ is($sql_result, '0',
+ "$type: filler column of pgbench_accounts has no NULL data");
+
$sql_result = $node->safe_psql('postgres',
'SELECT count(*) AS data_count FROM pgbench_history;');
- is($sql_result, '0', "$type: pgbench_history has no data");
+ is($sql_result, '0',
+ "$type: pgbench_history has no data");
}
# start a pgbench specific server
@@ -125,7 +130,7 @@ $node->pgbench(
'pgbench scale 1 initialization',);
# Check data state, after client-side data generation.
-check_data_state($node, 'client-side');
+check_data_state($node, 'client-side (default options)');
# Again, with all possible options
$node->pgbench(
@@ -143,6 +148,7 @@ $node->pgbench(
qr{done in \d+\.\d\d s }
],
'pgbench scale 1 initialization');
+check_data_state($node, 'client-side (all options)');
# Test interaction of --init-steps with legacy step-selection options
$node->pgbench(
@@ -164,6 +170,38 @@ $node->pgbench(
# Check data state, after server-side data generation.
check_data_state($node, 'server-side');
+# Test server-side generation with UNNEST
+$node->pgbench(
+ '--initialize --init-steps=dtI',
+ 0,
+ [qr{^$}],
+ [
+ qr{dropping old tables},
+ qr{creating tables},
+ qr{generating data \(server-side\)},
+ qr{done in \d+\.\d\d s }
+ ],
+ 'pgbench --init-steps server-side UNNEST');
+
+# Check data state, after server-side data generation.
+check_data_state($node, 'server-side (unnest)');
+
+# Test server-side generation with UNNEST
+$node->pgbench(
+ '--initialize --init-steps=dtC',
+ 0,
+ [qr{^$}],
+ [
+ qr{dropping old tables},
+ qr{creating tables},
+ qr{generating data \(client-side\)},
+ qr{done in \d+\.\d\d s }
+ ],
+ 'pgbench --init-steps client-side BINARY');
+
+# Check data state, after server-side data generation.
+check_data_state($node, 'client-side (binary)');
+
# Run all builtin scripts, for a few transactions each
$node->pgbench(
'--transactions=5 -Dfoo=bla --client=2 --protocol=simple --builtin=t'
--
2.43.0
From b3f2bce34232d299abc7644a8579f0ce49c8c9d6 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Sun, 23 Nov 2025 14:05:59 +0700
Subject: [PATCH v7 12/18] Fixing compiler warnings about unused procedures by
removing or commenting them out as they might be needed a bit later
---
src/bin/pgbench/pgbench.c | 31 +++++--------------------------
1 file changed, 5 insertions(+), 26 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 0a3ba21dcc9..682db61ff61 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -5201,7 +5201,7 @@ initGenerateDataClientSideText(PGconn *con)
/*
* Dumps binary buffer to file (purely for debugging)
- */
+ *
static void
dumpBufferToFile(char *filename)
{
@@ -5226,6 +5226,7 @@ dumpBufferToFile(char *filename)
fclose(file_ptr);
}
+ */
/*
* Save char data to buffer
@@ -5271,37 +5272,14 @@ addColumnCounter(int16_t n)
/*
* adds column with NULL value
- */
+ *
static void
addNullColumn()
{
int32_t null = -1;
bufferData((void *) &null, sizeof(null));
}
-
-/*
- * adds column with int8 value
*/
-static void
-addInt8Column(int8_t value)
-{
- int8_t data = value;
- int32_t size = sizeof(data);
- bufferData((void *) &size, sizeof(size));
- bufferData((void *) &data, sizeof(data));
-}
-
-/*
- * adds column with int16 value
- */
-static void
-addInt16Column(int16_t value)
-{
- int16_t data = value;
- int32_t size = sizeof(data);
- bufferData((void *) &size, sizeof(size));
- bufferData((void *) &data, sizeof(data));
-}
/*
* adds column with inti32 value
@@ -5329,7 +5307,7 @@ addInt64Column(int64_t value)
/*
* adds column with char value
- */
+ *
static void
addCharColumn(char *value)
{
@@ -5337,6 +5315,7 @@ addCharColumn(char *value)
bufferData((void *) &size, sizeof(size));
bufferCharData(value, size);
}
+ */
/*
* Starts communication with server for COPY FROM BINARY statement
--
2.43.0
From 9ab7fe302ba7de40593e7cba8ea1ca3b876c1ea5 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Thu, 29 Jan 2026 19:22:16 +0700
Subject: [PATCH v7 13/18] Moving PQclear call to the end of procedure to avoid
access to previously released memory
---
src/bin/pgbench/pgbench.c | 27 +++++++++++++++++----------
1 file changed, 17 insertions(+), 10 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 682db61ff61..3687b65871e 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -5363,11 +5363,11 @@ static void
initBranchBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
{
/*
- * Each row has following extra bytes:
+ * Each row of branch table has following extra bytes:
* - 2 bytes for number of columns
- * - 4 bytes as length for each column
+ * - 4 bytes as length for each of table's 3 columns
*/
- int16_t max_row_len = 35 + 2 + 4*3; /* max row size is 32 */
+ int16_t max_row_len = 35 + 2 + 4*3; /* max row size is 35 */
flushBuffer(con, res, max_row_len);
@@ -5384,9 +5384,9 @@ static void
initTellerBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
{
/*
- * Each row has following extra bytes:
+ * Each row of tellers table has following extra bytes:
* - 2 bytes for number of columns
- * - 4 bytes as length for each column
+ * - 4 bytes as length for each of table's 4 columns
*/
int16_t max_row_len = 40 + 2 + 4*4; /* max row size is 40 */
@@ -5406,9 +5406,9 @@ static void
initAccountBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
{
/*
- * Each row has following extra bytes:
+ * Each row of accounts table has following extra bytes:
* - 2 bytes for number of columns
- * - 4 bytes as length for each column
+ * - 4 bytes as length for each of table's 4 columns
*/
int16_t max_row_len = 250 + 2 + 4*4; /* max row size is 250 for int64 */
@@ -5455,11 +5455,11 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
if (PQresultStatus(res) != PGRES_COPY_IN)
pg_fatal("unexpected copy in result: %s", PQerrorMessage(con));
- PQclear(res);
sendBinaryCopyHeader(con);
+
for (int64_t i = start; i < start + base; i++)
{
init_row(con, res, i, base);
@@ -5470,8 +5470,10 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
else
fprintf(stderr, "Unexpected mode %d instead of %d\n", PQresultStatus(res), PGRES_COPY_IN);
+
sendBinaryCopyTrailer(con);
+
if (PQresultStatus(res) == PGRES_COPY_IN)
{
if (PQputCopyEnd(con, NULL) == 1) /* success */
@@ -5484,6 +5486,8 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
else
fprintf(stderr, "Error: %s\n", PQerrorMessage(con));
}
+
+ PQclear(res);
}
/*
@@ -5499,8 +5503,9 @@ initGenerateDataClientSideBinary(PGconn *con)
bin_copy_buffer_length = 0;
/*
- * we do all of this in one transaction to enable the backend's
- * data-loading optimizations
+ * we do all of this in multiple transactions
+ * to minimize load on DB server and perhaps
+ * in future allow load in parallel sessions
*/
executeStatement(con, "begin");
@@ -5511,6 +5516,8 @@ initGenerateDataClientSideBinary(PGconn *con)
for (int i = 0; i < scale; i++)
{
+ executeStatement(con, "begin");
+
initPopulateTableBinary(con, "pgbench_branches", "bid, bbalance",
i, nbranches, initBranchBinary);
initPopulateTableBinary(con, "pgbench_tellers", "tid, bid, tbalance",
--
2.43.0
From 3d7b2c53b3fa45b8848369bfbc49a0c7e5854304 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Thu, 29 Jan 2026 19:47:26 +0700
Subject: [PATCH v7 14/18] Fixing error about freeing memory twice
---
src/bin/pgbench/pgbench.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 3687b65871e..6f42438d970 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -5462,6 +5462,7 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
for (int64_t i = start; i < start + base; i++)
{
+ res = PQgetResult(con);
init_row(con, res, i, base);
}
@@ -5473,7 +5474,7 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
sendBinaryCopyTrailer(con);
-
+ res = PQgetResult(con);
if (PQresultStatus(res) == PGRES_COPY_IN)
{
if (PQputCopyEnd(con, NULL) == 1) /* success */
@@ -5481,7 +5482,6 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
res = PQgetResult(con);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
fprintf(stderr, "Error: %s\n", PQerrorMessage(con));
- PQclear(res);
}
else
fprintf(stderr, "Error: %s\n", PQerrorMessage(con));
--
2.43.0
From a71c90f556ba143335d9631ebbdf96b7120c6d1d Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Thu, 29 Jan 2026 20:24:54 +0700
Subject: [PATCH v7 15/18] Fixing memory leak during data flush in COPY BINARY
test
---
src/bin/pgbench/pgbench.c | 26 ++++++++++++++++----------
1 file changed, 16 insertions(+), 10 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 6f42438d970..8eca537b718 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -871,7 +871,7 @@ static bool socket_has_input(socket_set *sa, int fd, int idx);
/* callback used to build rows for COPY during data loading */
typedef void (*initRowMethod) (PQExpBufferData *sql, int64 curr);
-typedef void (*initRowMethodBin) (PGconn *con, PGresult *res, int64_t curr, int32_t parent);
+typedef void (*initRowMethodBin) (PGconn *con, int64_t curr, int32_t parent);
/* callback functions for our flex lexer */
static const PsqlScanCallbacks pgbench_callbacks = {
@@ -5345,13 +5345,18 @@ sendBinaryCopyTrailer(PGconn *con)
* Flashes current buffer over network if needed
*/
static void
-flushBuffer(PGconn *con, PGresult *res, int16_t row_len)
+flushBuffer(PGconn *con, int16_t row_len)
{
+ PGresult *res;
+
if (bin_copy_buffer_length + row_len > BIN_COPY_BUF_SIZE)
{
+ res = PQgetResult(con);
/* flush current buffer */
if (PQresultStatus(res) == PGRES_COPY_IN)
PQputCopyData(con, (char *) bin_copy_buffer, bin_copy_buffer_length);
+ else
+ pg_fatal("It is NOT a COPY command that is currently running");
bin_copy_buffer_length = 0;
}
}
@@ -5360,7 +5365,7 @@ flushBuffer(PGconn *con, PGresult *res, int16_t row_len)
* Sends current branch row to buffer
*/
static void
-initBranchBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
+initBranchBinary(PGconn *con, int64_t curr, int32_t parent)
{
/*
* Each row of branch table has following extra bytes:
@@ -5369,7 +5374,7 @@ initBranchBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
*/
int16_t max_row_len = 35 + 2 + 4*3; /* max row size is 35 */
- flushBuffer(con, res, max_row_len);
+ flushBuffer(con, max_row_len);
addColumnCounter(2);
@@ -5381,7 +5386,7 @@ initBranchBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
* Sends current teller row to buffer
*/
static void
-initTellerBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
+initTellerBinary(PGconn *con, int64_t curr, int32_t parent)
{
/*
* Each row of tellers table has following extra bytes:
@@ -5390,7 +5395,7 @@ initTellerBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
*/
int16_t max_row_len = 40 + 2 + 4*4; /* max row size is 40 */
- flushBuffer(con, res, max_row_len);
+ flushBuffer(con, max_row_len);
addColumnCounter(3);
@@ -5403,7 +5408,7 @@ initTellerBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
* Sends current account row to buffer
*/
static void
-initAccountBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
+initAccountBinary(PGconn *con, int64_t curr, int32_t parent)
{
/*
* Each row of accounts table has following extra bytes:
@@ -5412,7 +5417,7 @@ initAccountBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
*/
int16_t max_row_len = 250 + 2 + 4*4; /* max row size is 250 for int64 */
- flushBuffer(con, res, max_row_len);
+ flushBuffer(con, max_row_len);
addColumnCounter(3);
@@ -5462,10 +5467,10 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
for (int64_t i = start; i < start + base; i++)
{
- res = PQgetResult(con);
- init_row(con, res, i, base);
+ init_row(con, i, base);
}
+ res = PQgetResult(con);
if (PQresultStatus(res) == PGRES_COPY_IN)
PQputCopyData(con, (char *) bin_copy_buffer, bin_copy_buffer_length);
else
@@ -5474,6 +5479,7 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
sendBinaryCopyTrailer(con);
+
res = PQgetResult(con);
if (PQresultStatus(res) == PGRES_COPY_IN)
{
--
2.43.0
From a5933f40be0f3d408bb920f296fb5119c3ec28f4 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Fri, 30 Jan 2026 13:59:36 +0700
Subject: [PATCH v7 16/18] Fixing memory leak shown by valgrind
---
src/bin/pgbench/pgbench.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 8eca537b718..466d9023d72 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -5357,6 +5357,8 @@ flushBuffer(PGconn *con, int16_t row_len)
PQputCopyData(con, (char *) bin_copy_buffer, bin_copy_buffer_length);
else
pg_fatal("It is NOT a COPY command that is currently running");
+
+ PQclear(res);
bin_copy_buffer_length = 0;
}
}
@@ -5460,6 +5462,7 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
if (PQresultStatus(res) != PGRES_COPY_IN)
pg_fatal("unexpected copy in result: %s", PQerrorMessage(con));
+ PQclear(res);
sendBinaryCopyHeader(con);
@@ -5475,6 +5478,7 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
PQputCopyData(con, (char *) bin_copy_buffer, bin_copy_buffer_length);
else
fprintf(stderr, "Unexpected mode %d instead of %d\n", PQresultStatus(res), PGRES_COPY_IN);
+ PQclear(res);
sendBinaryCopyTrailer(con);
@@ -5485,6 +5489,7 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
{
if (PQputCopyEnd(con, NULL) == 1) /* success */
{
+ PQclear(res);
res = PQgetResult(con);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
fprintf(stderr, "Error: %s\n", PQerrorMessage(con));
@@ -5492,7 +5497,6 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
else
fprintf(stderr, "Error: %s\n", PQerrorMessage(con));
}
-
PQclear(res);
}
--
2.43.0
From 6a3036f898b050d5e71e70eaceccd63003aa6444 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Fri, 6 Feb 2026 13:53:07 +0700
Subject: [PATCH v7 17/18] Adding ability to switch data init between single
and multiple transactions
---
src/bin/pgbench/pgbench.c | 192 +++++++++----------
src/bin/pgbench/t/001_pgbench_with_server.pl | 93 ++++++++-
2 files changed, 173 insertions(+), 112 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 466d9023d72..af71e358e71 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -159,9 +159,8 @@ typedef struct socket_set
/********************************************************************
* some configurable parameters */
-
-#define DEFAULT_INIT_STEPS "dtgvp" /* default -I setting */
-#define ALL_INIT_STEPS "dtgCGiIvpf" /* all possible steps */
+#define DEFAULT_INIT_STEPS "dtgvp" /* default -I setting */
+#define ALL_INIT_STEPS "dtgMScGUvpf" /* all possible steps */
#define LOG_STEP_SECONDS 5 /* seconds between log messages */
#define DEFAULT_NXACTS 10 /* default nxacts */
@@ -171,14 +170,17 @@ typedef struct socket_set
#define MIN_ZIPFIAN_PARAM 1.001 /* minimum parameter for zipfian */
#define MAX_ZIPFIAN_PARAM 1000.0 /* maximum parameter for zipfian */
-/* original single transaction server-side method */
-#define GEN_TYPE_INSERT_ORIGINAL 'G' /* use INSERT .. SELECT generate_series to generate data */
-/* 'one transaction per scale' server-side methods */
-#define GEN_TYPE_INSERT_SERIES 'i' /* use INSERT .. SELECT generate_series to generate data */
-#define GEN_TYPE_INSERT_UNNEST 'I' /* use INSERT .. SELECT unnest to generate data */
-#define GEN_TYPE_COPY_ORIGINAL 'g' /* use COPY .. FROM STDIN .. TEXT to generate data */
-#define GEN_TYPE_COPY_BINARY 'C' /* use COPY .. FROM STDIN .. BINARY to generate data */
-
+/* server-side methods to generate data */
+#define INIT_STEP_GEN_TYPE_INSERT_SERIES 'G' /* use INSERT .. SELECT generate_series to generate data */
+#define INIT_STEP_GEN_TYPE_INSERT_UNNEST 'U' /* use INSERT .. SELECT unnest to generate data */
+/* client-side methods to generate data */
+#define INIT_STEP_GEN_TYPE_COPY_TEXT 'g' /* use COPY .. FROM STDIN .. TEXT to generate data */
+#define INIT_STEP_GEN_TYPE_COPY_BINARY 'c' /* use COPY .. FROM STDIN .. BINARY to generate data */
+/* data init pseudo steps */
+#define INIT_STEP_GEN_TYPE_SINGLE_XACT 'S' /* switch to init data as single transaction */
+#define INIT_STEP_GEN_TYPE_MULTI_XACT 'M' /* switch to init data as multiple transactions */
+
+static bool multi_xact = false; /* init data type (as single or multiple transactions) */
static int nxacts = 0; /* number of transactions per client */
static int duration = 0; /* duration in seconds */
static int64 end_time = 0; /* when to stop in micro seconds, under -T */
@@ -192,7 +194,7 @@ static int scale = 1;
/*
* mode of data generation to use
*/
-static char data_generation_type = GEN_TYPE_COPY_ORIGINAL;
+static char data_generation_type = INIT_STEP_GEN_TYPE_COPY_TEXT;
/*
* COPY FROM BINARY execution buffer
@@ -934,11 +936,13 @@ usage(void)
" run selected initialization steps, in the specified order\n"
" d: drop any existing pgbench tables\n"
" t: create the tables used by the standard pgbench scenario\n"
- " g: generate data, client-side\n"
- " C: client-side (single TNX) COPY .. FROM STDIN .. BINARY\n"
- " G: generate data, server-side in single transaction\n"
- " i: server-side (multiple TXNs) INSERT .. SELECT generate_series\n"
- " I: server-side (multiple TXNs) INSERT .. SELECT unnest\n"
+ " to generate data, client-side:\n"
+ " g: COPY .. FROM STDIN .. TEXT\n"
+ " c: COPY .. FROM STDIN .. BINARY\n"
+ " to generate data, server-side:\n"
+ " G: INSERT .. SELECT generate_series\n"
+ " U: INSERT .. SELECT unnest\n"
+ " M: use multiple transactions to initialize data\n"
" v: invoke VACUUM on the standard tables\n"
" p: create primary key indexes on the standard tables\n"
" f: create foreign keys between the standard tables\n"
@@ -5049,8 +5053,8 @@ initAccount(PQExpBufferData *sql, int64 curr)
}
static void
-initPopulateTable(PGconn *con, const char *table, int64 base,
- initRowMethod init_row)
+initPopulateTableText(PGconn *con, const char *table, int64 base,
+ initRowMethod init_row)
{
int n;
int64 k;
@@ -5178,6 +5182,9 @@ initGenerateDataClientSideText(PGconn *con)
{
fprintf(stderr, "TEXT mode...\n");
+ if (multi_xact)
+ fprintf(stderr, "WARNING! Multiple transactions are not supported in this mode\n");
+
/*
* we do all of this in one transaction to enable the backend's
* data-loading optimizations
@@ -5191,9 +5198,9 @@ initGenerateDataClientSideText(PGconn *con)
* fill branches, tellers, accounts in that order in case foreign keys
* already exist
*/
- initPopulateTable(con, "pgbench_branches", nbranches, initBranch);
- initPopulateTable(con, "pgbench_tellers", ntellers, initTeller);
- initPopulateTable(con, "pgbench_accounts", naccounts, initAccount);
+ initPopulateTableText(con, "pgbench_branches", nbranches, initBranch);
+ initPopulateTableText(con, "pgbench_tellers", ntellers, initTeller);
+ initPopulateTableText(con, "pgbench_accounts", naccounts, initAccount);
executeStatement(con, "commit");
}
@@ -5450,7 +5457,8 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
/* Use COPY with FREEZE on v14 and later for all ordinary tables */
if ((PQserverVersion(con) >= 140000) &&
get_table_relkind(con, table) == RELKIND_RELATION)
- copy_statement_fmt = "copy %s (%s) from stdin with (format binary)";
+ if (!multi_xact)
+ copy_statement_fmt = "copy %s (%s) from stdin with (format binary, freeze on)";
n = pg_snprintf(copy_statement, sizeof(copy_statement), copy_statement_fmt, table, columns);
if (n >= sizeof(copy_statement))
@@ -5522,11 +5530,13 @@ initGenerateDataClientSideBinary(PGconn *con)
/* truncate away any old data */
initTruncateTables(con);
- executeStatement(con, "commit");
+ if (multi_xact)
+ executeStatement(con, "commit");
for (int i = 0; i < scale; i++)
{
- executeStatement(con, "begin");
+ if (multi_xact)
+ executeStatement(con, "begin");
initPopulateTableBinary(con, "pgbench_branches", "bid, bbalance",
i, nbranches, initBranchBinary);
@@ -5535,9 +5545,13 @@ initGenerateDataClientSideBinary(PGconn *con)
initPopulateTableBinary(con, "pgbench_accounts", "aid, bid, abalance",
i, naccounts, initAccountBinary);
- executeStatement(con, "commit");
+ if (multi_xact)
+ executeStatement(con, "commit");
}
+ if (!multi_xact)
+ executeStatement(con, "commit");
+
pg_free(bin_copy_buffer);
}
@@ -5547,14 +5561,15 @@ initGenerateDataClientSideBinary(PGconn *con)
static void
initGenerateDataClientSide(PGconn *con)
{
- fprintf(stderr, "generating data (client-side) in ");
+ fprintf(stderr, "generating data (client-side as %s transaction%s) in ",
+ multi_xact ? "multiple" : "single", multi_xact ? "s" : "");
switch (data_generation_type)
{
- case GEN_TYPE_COPY_ORIGINAL:
+ case INIT_STEP_GEN_TYPE_COPY_TEXT:
initGenerateDataClientSideText(con);
break;
- case GEN_TYPE_COPY_BINARY:
+ case INIT_STEP_GEN_TYPE_COPY_BINARY:
initGenerateDataClientSideBinary(con);
break;
}
@@ -5562,58 +5577,7 @@ initGenerateDataClientSide(PGconn *con)
/*
* Generating data via INSERT .. SELECT .. FROM generate_series
- * whole dataset in single transaction
- */
-static void
-generateDataInsertSingleTXN(PGconn *con)
-{
- PQExpBufferData sql;
-
- fprintf(stderr, "via INSERT .. SELECT generate_series... in single TXN\n");
-
-
- /*
- * we do all of this in one transaction to enable the backend's
- * data-loading optimizations
- */
- executeStatement(con, "begin");
-
- /* truncate away any old data */
- initTruncateTables(con);
-
- initPQExpBuffer(&sql);
-
- printfPQExpBuffer(&sql,
- "insert into pgbench_branches(bid, bbalance) "
- "select bid, 0 "
- "from generate_series(1, %d) as bid",
- scale * nbranches);
- executeStatement(con, sql.data);
-
- printfPQExpBuffer(&sql,
- "insert into pgbench_tellers(tid, bid, tbalance) "
- "select tid + 1, tid / %d + 1, 0 "
- "from generate_series(0, %d) as tid",
- ntellers, (scale * ntellers) - 1);
- executeStatement(con, sql.data);
-
- printfPQExpBuffer(&sql,
- "insert into pgbench_accounts(aid, bid, abalance, "
- "filler) "
- "select aid + 1, aid / %d + 1, 0, '' "
- "from generate_series(0, " INT64_FORMAT ") as aid",
- naccounts, (int64) (scale * naccounts) - 1);
- executeStatement(con, sql.data);
-
- executeStatement(con, "commit");
-
- termPQExpBuffer(&sql);
-}
-
-
-/*
- * Generating data via INSERT .. SELECT .. FROM generate_series
- * One transaction per 'scale'
+ * Possibly as "One transaction per scale" in multi-transaction mode
*/
static void
generateDataInsertSeries(PGconn *con)
@@ -5629,11 +5593,13 @@ generateDataInsertSeries(PGconn *con)
/* truncate away any old data */
initTruncateTables(con);
- executeStatement(con, "commit");
+ if (multi_xact)
+ executeStatement(con, "commit");
for (int i = 0; i < scale; i++)
{
- executeStatement(con, "begin");
+ if (multi_xact)
+ executeStatement(con, "begin");
printfPQExpBuffer(&sql,
"insert into pgbench_branches(bid, bbalance) "
@@ -5657,15 +5623,19 @@ generateDataInsertSeries(PGconn *con)
(int64) (i + 1) * naccounts - 1);
executeStatement(con, sql.data);
- executeStatement(con, "commit");
+ if (multi_xact)
+ executeStatement(con, "commit");
}
+ if (!multi_xact)
+ executeStatement(con, "commit");
+
termPQExpBuffer(&sql);
}
/*
* Generating data via INSERT .. SELECT .. FROM unnest
- * One transaction per 'scale'
+ * Possibly as "One transaction per scale" in multi-tansaction mode
*/
static void
generateDataInsertUnnest(PGconn *con)
@@ -5681,11 +5651,13 @@ generateDataInsertUnnest(PGconn *con)
/* truncate away any old data */
initTruncateTables(con);
- executeStatement(con, "commit");
+ if (multi_xact)
+ executeStatement(con, "commit");
for (int s = 0; s < scale; s++)
{
- executeStatement(con, "begin");
+ if (multi_xact)
+ executeStatement(con, "begin");
printfPQExpBuffer(&sql,
"insert into pgbench_branches(bid,bbalance) "
@@ -5714,14 +5686,18 @@ generateDataInsertUnnest(PGconn *con)
(int64) (s + 1) * naccounts, naccounts);
executeStatement(con, sql.data);
- executeStatement(con, "commit");
+ if (multi_xact)
+ executeStatement(con, "commit");
}
+ if (!multi_xact)
+ executeStatement(con, "commit");
+
termPQExpBuffer(&sql);
}
/*
- * Fill the standard tables with some data generated on the server
+ * Fill the standard tables with some data generated on the server side
*
* As already the case with the client-side data generation, the filler
* column defaults to NULL in pgbench_branches and pgbench_tellers,
@@ -5730,17 +5706,15 @@ generateDataInsertUnnest(PGconn *con)
static void
initGenerateDataServerSide(PGconn *con)
{
- fprintf(stderr, "generating data (server-side) ");
+ fprintf(stderr, "generating data (server-side as %s transaction%s) ",
+ multi_xact ? "multiple" : "single", multi_xact ? "s" : "");
switch (data_generation_type)
{
- case GEN_TYPE_INSERT_ORIGINAL:
- generateDataInsertSingleTXN(con);
- break;
- case GEN_TYPE_INSERT_SERIES:
+ case INIT_STEP_GEN_TYPE_INSERT_SERIES:
generateDataInsertSeries(con);
break;
- case GEN_TYPE_INSERT_UNNEST:
+ case INIT_STEP_GEN_TYPE_INSERT_UNNEST:
generateDataInsertUnnest(con);
break;
}
@@ -5845,19 +5819,20 @@ checkInitSteps(const char *initialize_steps)
switch (*step)
{
- case 'g':
- case 'C':
- case 'G':
- case 'i':
- case 'I':
+ case INIT_STEP_GEN_TYPE_COPY_TEXT:
+ case INIT_STEP_GEN_TYPE_COPY_BINARY:
+ case INIT_STEP_GEN_TYPE_INSERT_SERIES:
+ case INIT_STEP_GEN_TYPE_INSERT_UNNEST:
data_init_type++;
data_generation_type = *step;
break;
}
}
+ if (data_init_type == 0)
+ pg_log_error("WARNING! No data generation type is provided");
if (data_init_type > 1)
- pg_log_error("WARNING! More than one type of server-side data generation is requested");
+ pg_log_error("WARNING! More than one type of data initialization is requested");
}
/*
@@ -5895,17 +5870,22 @@ runInitSteps(const char *initialize_steps)
op = "create tables";
initCreateTables(con);
break;
- case 'g':
- case 'C':
+ case INIT_STEP_GEN_TYPE_COPY_TEXT:
+ case INIT_STEP_GEN_TYPE_COPY_BINARY:
op = "client-side generate";
initGenerateDataClientSide(con);
break;
- case 'G':
- case 'i':
- case 'I':
+ case INIT_STEP_GEN_TYPE_INSERT_SERIES:
+ case INIT_STEP_GEN_TYPE_INSERT_UNNEST:
op = "server-side generate";
initGenerateDataServerSide(con);
break;
+ case INIT_STEP_GEN_TYPE_SINGLE_XACT:
+ multi_xact = false;
+ break;
+ case INIT_STEP_GEN_TYPE_MULTI_XACT:
+ multi_xact = true;
+ break;
case 'v':
op = "vacuum";
initVacuum(con);
diff --git a/src/bin/pgbench/t/001_pgbench_with_server.pl b/src/bin/pgbench/t/001_pgbench_with_server.pl
index a377048ead1..acb1e31d3e8 100644
--- a/src/bin/pgbench/t/001_pgbench_with_server.pl
+++ b/src/bin/pgbench/t/001_pgbench_with_server.pl
@@ -117,6 +117,7 @@ $node->pgbench(
[qr{Perhaps you need to do initialization}],
'run without init');
+
# Initialize pgbench tables scale 1
$node->pgbench(
'-i', 0,
@@ -160,7 +161,7 @@ $node->pgbench(
qr{creating tables},
qr{creating 3 partitions},
qr{creating primary keys},
- qr{generating data \(server-side\)},
+ qr{generating data \(server-side as single transaction\)},
qr{creating foreign keys},
qr{(?!vacuuming)}, # no vacuum
qr{done in \d+\.\d\d s }
@@ -170,15 +171,16 @@ $node->pgbench(
# Check data state, after server-side data generation.
check_data_state($node, 'server-side');
+
# Test server-side generation with UNNEST
$node->pgbench(
- '--initialize --init-steps=dtI',
+ '--initialize --init-steps=dtU',
0,
[qr{^$}],
[
qr{dropping old tables},
qr{creating tables},
- qr{generating data \(server-side\)},
+ qr{generating data \(server-side as single transaction\)},
qr{done in \d+\.\d\d s }
],
'pgbench --init-steps server-side UNNEST');
@@ -186,15 +188,48 @@ $node->pgbench(
# Check data state, after server-side data generation.
check_data_state($node, 'server-side (unnest)');
-# Test server-side generation with UNNEST
+
+# Test server-side generation with COPY TEXT
+$node->pgbench(
+ '--initialize --init-steps=dtg',
+ 0,
+ [qr{^$}],
+ [
+ qr{dropping old tables},
+ qr{creating tables},
+ qr{generating data \(client-side as single transaction},
+ qr{done in \d+\.\d\d s }
+ ],
+ 'pgbench --init-steps client-side TEXT');
+
+# Check data state, after server-side data generation.
+check_data_state($node, 'client-side (text)');
+
+$node->pgbench(
+ '--initialize --init-steps=dtMg',
+ 0,
+ [qr{^$}],
+ [
+ qr{dropping old tables},
+ qr{creating tables},
+ qr{generating data \(client-side as multiple transactions},
+ qr{done in \d+\.\d\d s }
+ ],
+ 'pgbench --init-steps client-side TEXT');
+
+# Check data state, after server-side data generation.
+check_data_state($node, 'client-side (text)');
+
+
+# Test server-side generation with COPY BINARY
$node->pgbench(
- '--initialize --init-steps=dtC',
+ '--initialize --init-steps=dtc',
0,
[qr{^$}],
[
qr{dropping old tables},
qr{creating tables},
- qr{generating data \(client-side\)},
+ qr{generating data \(client-side as single transaction},
qr{done in \d+\.\d\d s }
],
'pgbench --init-steps client-side BINARY');
@@ -202,6 +237,52 @@ $node->pgbench(
# Check data state, after server-side data generation.
check_data_state($node, 'client-side (binary)');
+$node->pgbench(
+ '--initialize --init-steps=dtSc',
+ 0,
+ [qr{^$}],
+ [
+ qr{dropping old tables},
+ qr{creating tables},
+ qr{generating data \(client-side as single transaction},
+ qr{done in \d+\.\d\d s }
+ ],
+ 'pgbench --init-steps client-side BINARY');
+
+# Check data state, after server-side data generation.
+check_data_state($node, 'client-side (binary)');
+
+$node->pgbench(
+ '--initialize --init-steps=dtMc',
+ 0,
+ [qr{^$}],
+ [
+ qr{dropping old tables},
+ qr{creating tables},
+ qr{generating data \(client-side as multiple transactions},
+ qr{done in \d+\.\d\d s }
+ ],
+ 'pgbench --init-steps client-side BINARY');
+
+# Check data state, after server-side data generation.
+check_data_state($node, 'client-side (binary)');
+
+$node->pgbench(
+ '--initialize --init-steps=dtMc',
+ 0,
+ [qr{^$}],
+ [
+ qr{dropping old tables},
+ qr{creating tables},
+ qr{generating data \(client-side as multiple transactions},
+ qr{done in \d+\.\d\d s }
+ ],
+ 'pgbench --init-steps client-side BINARY');
+
+# Check data state, after server-side data generation.
+check_data_state($node, 'client-side (binary)');
+
+
# Run all builtin scripts, for a few transactions each
$node->pgbench(
'--transactions=5 -Dfoo=bla --client=2 --protocol=simple --builtin=t'
--
2.43.0
From 2471bb47bf715402ece61b084b69afe3bb53742e Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Fri, 6 Feb 2026 13:55:47 +0700
Subject: [PATCH v7 18/18] Removing commented out procedures
---
src/bin/pgbench/pgbench.c | 52 ---------------------------------------
1 file changed, 52 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index af71e358e71..34464b7037c 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -5206,35 +5206,6 @@ initGenerateDataClientSideText(PGconn *con)
}
-/*
- * Dumps binary buffer to file (purely for debugging)
- *
-static void
-dumpBufferToFile(char *filename)
-{
- FILE *file_ptr;
- size_t bytes_written;
-
- file_ptr = fopen(filename, "wb");
- if (file_ptr == NULL)
- {
- fprintf(stderr, "Error opening file %s\n", filename);
- return; // EXIT_FAILURE;
- }
-
- bytes_written = fwrite(bin_copy_buffer, 1, bin_copy_buffer_length, file_ptr);
-
- if (bytes_written != bin_copy_buffer_length)
- {
- fprintf(stderr, "Error writing to file or incomplete write\n");
- fclose(file_ptr);
- return; // EXIT_FAILURE;
- }
-
- fclose(file_ptr);
-}
- */
-
/*
* Save char data to buffer
*/
@@ -5277,17 +5248,6 @@ addColumnCounter(int16_t n)
bufferData((void *) &n, sizeof(n));
}
-/*
- * adds column with NULL value
- *
-static void
-addNullColumn()
-{
- int32_t null = -1;
- bufferData((void *) &null, sizeof(null));
-}
- */
-
/*
* adds column with inti32 value
*/
@@ -5312,18 +5272,6 @@ addInt64Column(int64_t value)
bufferData((void *) &data, sizeof(data));
}
-/*
- * adds column with char value
- *
-static void
-addCharColumn(char *value)
-{
- int32_t size = strlen(value);
- bufferData((void *) &size, sizeof(size));
- bufferCharData(value, size);
-}
- */
-
/*
* Starts communication with server for COPY FROM BINARY statement
*/
--
2.43.0
^ permalink raw reply [nested|flat] 98+ messages in thread
* RE: Idea to enhance pgbench by more modes to generate data (multi-TXNs, UNNEST, COPY BINARY)
2026-01-30 13:22 RE: Idea to enhance pgbench by more modes to generate data (multi-TXNs, UNNEST, COPY BINARY) Madyshev Egor <[email protected]>
2026-01-30 14:47 ` Re: Idea to enhance pgbench by more modes to generate data (multi-TXNs, UNNEST, COPY BINARY) Boris Mironov <[email protected]>
2026-02-06 10:48 ` Re: Idea to enhance pgbench by more modes to generate data (multi-TXNs, UNNEST, COPY BINARY) Boris Mironov <[email protected]>
2026-02-06 11:08 ` Re: Idea to enhance pgbench by more modes to generate data (multi-TXNs, UNNEST, COPY BINARY) Boris Mironov <[email protected]>
@ 2026-02-09 13:19 ` Madyshev Egor <[email protected]>
2026-02-23 10:41 ` Re: Idea to enhance pgbench by more modes to generate data (multi-TXNs, UNNEST, COPY BINARY) Boris Mironov <[email protected]>
0 siblings, 1 reply; 98+ messages in thread
From: Madyshev Egor @ 2026-02-09 13:19 UTC (permalink / raw)
To: Boris Mironov <[email protected]>; pgsql-hackers
Hello, Boris
Thank you for making new productivity measurements. I`ve reviewed the v7
version and have some feedback below.
1. Perhaps we should rename functions initRowMethodBin to
initRowMethodBinary, initPopulateTableText to initPopulateTableTextCopy,
initPopulateTableBinary to initPopulateTableBinaryCopy,
initGenerateDataClientSideText to initGenerateDataClientSideTextFrmt,
initGenerateDataClientSideBinary to initGenerateDataClientSideBinaryFrmt
for better clarity.
2. The --help output currently describes only the 'M' mode. Should we
also add a description for the 'S' mode for completeness?
3. I`m wondering if the default value for the 'filler' column in
initCreateTables is necessary? The current functionality seems
unaffected, so perhaps we could avoid changing this function to keep the
diff minimal.
4. I noticed that vanilla client functionality in M mode is not
implemented. Is there a specific reason for this? It seems feasible to
implement by passing a counter, similar to how it`s done in
initPopulateTableBinary. If there are reasons not to implement it, in
mode Mg pgbench should not run mode Sg, it just pg_fatal.
5. In mode client binary format generation c, It would be the right
thing to do implement write progress of generating data and 'quiet' mode,
as it already implement in client text format generation g.
6. In bufferData, when len == 1, we call bufferCharData, which already
increments bin_copy_buffer_length. However, at the end of bufferCharData
we increment it again, leading to a double increment.
7. I suggest adding column count in function initPopulateTableBinary,
and initBranchBinary, initTellerBinary, initAccountBinary, to pass it
from initAccountBinary to init_row, and use it in functions
initBranchBinary, initTellerBinary, initAccountBinary. This will
increase the readability of the code, and remove the magic numbers in
the addColumnCounter call.
8. I think check and install data_generation_type in function
checkInitSteps is not quite right. In the current realization, pgbench
allows run data generation many times (dtCdtC...), so i suggest do not
touching this functionality. My suggestion would be to revert all the
changes from function checkInitSteps, set the data_generation_type in
switch in runInitSteps and remove call function checkInitSteps
from main.
Best regards,
Egor
^ permalink raw reply [nested|flat] 98+ messages in thread
* Re: Idea to enhance pgbench by more modes to generate data (multi-TXNs, UNNEST, COPY BINARY)
2026-01-30 13:22 RE: Idea to enhance pgbench by more modes to generate data (multi-TXNs, UNNEST, COPY BINARY) Madyshev Egor <[email protected]>
2026-01-30 14:47 ` Re: Idea to enhance pgbench by more modes to generate data (multi-TXNs, UNNEST, COPY BINARY) Boris Mironov <[email protected]>
2026-02-06 10:48 ` Re: Idea to enhance pgbench by more modes to generate data (multi-TXNs, UNNEST, COPY BINARY) Boris Mironov <[email protected]>
2026-02-06 11:08 ` Re: Idea to enhance pgbench by more modes to generate data (multi-TXNs, UNNEST, COPY BINARY) Boris Mironov <[email protected]>
2026-02-09 13:19 ` RE: Idea to enhance pgbench by more modes to generate data (multi-TXNs, UNNEST, COPY BINARY) Madyshev Egor <[email protected]>
@ 2026-02-23 10:41 ` Boris Mironov <[email protected]>
0 siblings, 0 replies; 98+ messages in thread
From: Boris Mironov @ 2026-02-23 10:41 UTC (permalink / raw)
To: Madyshev Egor <[email protected]>; pgsql-hackers
Hi Egor,
Here comes v8 of the patch with all requested changes. Sorry it took a little while to implement...
> 1. Perhaps we should rename functions initRowMethodBin to
> initRowMethodBinary, initPopulateTableText to initPopulateTableTextCopy,
> initPopulateTableBinary to initPopulateTableBinaryCopy,
> initGenerateDataClientSideText to initGenerateDataClientSideTextFrmt,
> initGenerateDataClientSideBinary to initGenerateDataClientSideBinaryFrmt
> for better clarity.
Done
> 2. The --help output currently describes only the 'M' mode. Should we
>also add a description for the 'S' mode for completeness?
Fixed. Thanks for noticing
> 3. I`m wondering if the default value for the 'filler' column in
> initCreateTables is necessary? The current functionality seems
> unaffected, so perhaps we could avoid changing this function to keep the
> diff minimal.
That was done purely for debugging. Fixed
> 4. I noticed that vanilla client functionality in M mode is not
> implemented. Is there a specific reason for this? It seems feasible to
> implement by passing a counter, similar to how it`s done in
> initPopulateTableBinary. If there are reasons not to implement it, in
> mode Mg pgbench should not run mode Sg, it just pg_fatal.
I didn't do it initially to keep original logic intact. To unify new patch
with existing codebase I've added multi-transactional functionality
to client-side data generator and extracted "show progress" code
into own procedure. Now both client-side generators use it.
> 5. In mode client binary format generation c, It would be the right
> thing to do implement write progress of generating data and 'quiet' mode,
> as it already implement in client text format generation g.
Everything is unified now via new procedure showPopulateTableCopyProgress
> 6. In bufferData, when len == 1, we call bufferCharData, which already
> increments bin_copy_buffer_length. However, at the end of bufferCharData
> we increment it again, leading to a double increment.
Fixed. Thank you
> 7. I suggest adding column count in function initPopulateTableBinary,
> and initBranchBinary, initTellerBinary, initAccountBinary, to pass it
> from initAccountBinary to init_row, and use it in functions
> initBranchBinary, initTellerBinary, initAccountBinary. This will
> increase the readability of the code, and remove the magic numbers in
> the addColumnCounter call.
Done
> 8. I think check and install data_generation_type in function
> checkInitSteps is not quite right. In the current realization, pgbench
> allows run data generation many times (dtCdtC...), so i suggest do not
> touching this functionality. My suggestion would be to revert all the
> changes from function checkInitSteps, set the data_generation_type in
> switch in runInitSteps and remove call function checkInitSteps
> from main.
I thought about such solution initially, but then decided to implement extra
procedure where I can count how many times we asked to initialize data.
Since this is legacy code and I can see need for ability to run init multiple
times (eg, get exact timing of different methods for comparison or run
same code multiple times to get average timing), I just show warning.
All in all, thanks a lot for your deep analysis of the code and great
suggestions on how to make better.
Best regards,
Boris
Attachments:
[application/octet-stream] v8-pgbench-faster-modes.patch (116.4K, ../../PH0PR08MB70208182284EBA18F64228798877A@PH0PR08MB7020.namprd08.prod.outlook.com/3-v8-pgbench-faster-modes.patch)
download | inline diff:
From c768f399c556295de7d53895410e686d86b4b960 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Sun, 9 Nov 2025 19:34:58 +0700
Subject: [PATCH v8 01/23] Converting one huge transaction into series of one
per 'scale'
---
src/bin/pgbench/pgbench.c | 61 ++++++++++++++++++++++++++-------------
1 file changed, 41 insertions(+), 20 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index d8764ba6fe0..284a7c860f1 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -181,6 +181,12 @@ static int64 end_time = 0; /* when to stop in micro seconds, under -T */
*/
static int scale = 1;
+/*
+ * scaling factor after which we switch to multiple transactions during
+ * data population phase on server side
+ */
+static int64 single_txn_scale_limit = 1;
+
/*
* fillfactor. for example, fillfactor = 90 will use only 90 percent
* space during inserts and leave 10 percent free.
@@ -5213,6 +5219,7 @@ static void
initGenerateDataServerSide(PGconn *con)
{
PQExpBufferData sql;
+ int chunk = (scale >= single_txn_scale_limit) ? 1 : scale;
fprintf(stderr, "generating data (server-side)...\n");
@@ -5225,30 +5232,44 @@ initGenerateDataServerSide(PGconn *con)
/* truncate away any old data */
initTruncateTables(con);
+ executeStatement(con, "commit");
+
initPQExpBuffer(&sql);
- printfPQExpBuffer(&sql,
- "insert into pgbench_branches(bid,bbalance) "
- "select bid, 0 "
- "from generate_series(1, %d) as bid", nbranches * scale);
- executeStatement(con, sql.data);
-
- printfPQExpBuffer(&sql,
- "insert into pgbench_tellers(tid,bid,tbalance) "
- "select tid, (tid - 1) / %d + 1, 0 "
- "from generate_series(1, %d) as tid", ntellers, ntellers * scale);
- executeStatement(con, sql.data);
-
- printfPQExpBuffer(&sql,
- "insert into pgbench_accounts(aid,bid,abalance,filler) "
- "select aid, (aid - 1) / %d + 1, 0, '' "
- "from generate_series(1, " INT64_FORMAT ") as aid",
- naccounts, (int64) naccounts * scale);
- executeStatement(con, sql.data);
+ for (int i = 0; i < scale; i += chunk) {
+ executeStatement(con, "begin");
+
+ printfPQExpBuffer(&sql,
+ "insert into pgbench_branches(bid,bbalance) "
+ "select bid + 1, 0 "
+ "from generate_series(%d, %d) as bid", i, i + chunk);
+ //"select bid, 0 "
+ //"from generate_series(1, %d) as bid", nbranches * scale);
+ executeStatement(con, sql.data);
+
+ printfPQExpBuffer(&sql,
+ "insert into pgbench_tellers(tid,bid,tbalance) "
+ "select tid + 1, tid / %d + 1, 0 "
+ "from generate_series(%d, %d) as tid",
+ ntellers, i * ntellers, (i + chunk) * ntellers - 1);
+ //"select tid, (tid - 1) / %d + 1, 0 "
+ //"from generate_series(1, %d) as tid", ntellers, ntellers * scale);
+ executeStatement(con, sql.data);
+
+ printfPQExpBuffer(&sql,
+ "insert into pgbench_accounts(aid,bid,abalance,filler) "
+ "select aid + 1, aid / %d + 1, 0, '' "
+ "from generate_series(" INT64_FORMAT ", " INT64_FORMAT ") as aid",
+ naccounts, (int64) i * naccounts, (int64) (i + chunk) * naccounts - 1);
+ //"select aid, (aid - 1) / %d + 1, 0, '' "
+ //"from generate_series(1, " INT64_FORMAT ") as aid",
+ //naccounts, (int64) naccounts * scale);
+ executeStatement(con, sql.data);
+
+ executeStatement(con, "commit");
+ }
termPQExpBuffer(&sql);
-
- executeStatement(con, "commit");
}
/*
--
2.43.0
From 0eddb156c187d829c4381bc928c5314705928852 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Sun, 9 Nov 2025 20:13:23 +0700
Subject: [PATCH v8 02/23] Getting rid off limit for single transaction size
during data generation
---
src/bin/pgbench/pgbench.c | 15 ++++-----------
1 file changed, 4 insertions(+), 11 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 284a7c860f1..28b72e4cf1f 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -181,12 +181,6 @@ static int64 end_time = 0; /* when to stop in micro seconds, under -T */
*/
static int scale = 1;
-/*
- * scaling factor after which we switch to multiple transactions during
- * data population phase on server side
- */
-static int64 single_txn_scale_limit = 1;
-
/*
* fillfactor. for example, fillfactor = 90 will use only 90 percent
* space during inserts and leave 10 percent free.
@@ -5219,7 +5213,6 @@ static void
initGenerateDataServerSide(PGconn *con)
{
PQExpBufferData sql;
- int chunk = (scale >= single_txn_scale_limit) ? 1 : scale;
fprintf(stderr, "generating data (server-side)...\n");
@@ -5236,13 +5229,13 @@ initGenerateDataServerSide(PGconn *con)
initPQExpBuffer(&sql);
- for (int i = 0; i < scale; i += chunk) {
+ for (int i = 0; i < scale; i++) {
executeStatement(con, "begin");
printfPQExpBuffer(&sql,
"insert into pgbench_branches(bid,bbalance) "
"select bid + 1, 0 "
- "from generate_series(%d, %d) as bid", i, i + chunk);
+ "from generate_series(%d, %d) as bid", i, i + 1);
//"select bid, 0 "
//"from generate_series(1, %d) as bid", nbranches * scale);
executeStatement(con, sql.data);
@@ -5251,7 +5244,7 @@ initGenerateDataServerSide(PGconn *con)
"insert into pgbench_tellers(tid,bid,tbalance) "
"select tid + 1, tid / %d + 1, 0 "
"from generate_series(%d, %d) as tid",
- ntellers, i * ntellers, (i + chunk) * ntellers - 1);
+ ntellers, i * ntellers, (i + 1) * ntellers - 1);
//"select tid, (tid - 1) / %d + 1, 0 "
//"from generate_series(1, %d) as tid", ntellers, ntellers * scale);
executeStatement(con, sql.data);
@@ -5260,7 +5253,7 @@ initGenerateDataServerSide(PGconn *con)
"insert into pgbench_accounts(aid,bid,abalance,filler) "
"select aid + 1, aid / %d + 1, 0, '' "
"from generate_series(" INT64_FORMAT ", " INT64_FORMAT ") as aid",
- naccounts, (int64) i * naccounts, (int64) (i + chunk) * naccounts - 1);
+ naccounts, (int64) i * naccounts, (int64) (i + 1) * naccounts - 1);
//"select aid, (aid - 1) / %d + 1, 0, '' "
//"from generate_series(1, " INT64_FORMAT ") as aid",
//naccounts, (int64) naccounts * scale);
--
2.43.0
From c5659cf474ec273c057668f30a4f435fd02f2da7 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Sun, 9 Nov 2025 20:38:36 +0700
Subject: [PATCH v8 03/23] No need to keep old code in comments
---
src/bin/pgbench/pgbench.c | 7 -------
1 file changed, 7 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 28b72e4cf1f..97895aa9edf 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -5236,8 +5236,6 @@ initGenerateDataServerSide(PGconn *con)
"insert into pgbench_branches(bid,bbalance) "
"select bid + 1, 0 "
"from generate_series(%d, %d) as bid", i, i + 1);
- //"select bid, 0 "
- //"from generate_series(1, %d) as bid", nbranches * scale);
executeStatement(con, sql.data);
printfPQExpBuffer(&sql,
@@ -5245,8 +5243,6 @@ initGenerateDataServerSide(PGconn *con)
"select tid + 1, tid / %d + 1, 0 "
"from generate_series(%d, %d) as tid",
ntellers, i * ntellers, (i + 1) * ntellers - 1);
- //"select tid, (tid - 1) / %d + 1, 0 "
- //"from generate_series(1, %d) as tid", ntellers, ntellers * scale);
executeStatement(con, sql.data);
printfPQExpBuffer(&sql,
@@ -5254,9 +5250,6 @@ initGenerateDataServerSide(PGconn *con)
"select aid + 1, aid / %d + 1, 0, '' "
"from generate_series(" INT64_FORMAT ", " INT64_FORMAT ") as aid",
naccounts, (int64) i * naccounts, (int64) (i + 1) * naccounts - 1);
- //"select aid, (aid - 1) / %d + 1, 0, '' "
- //"from generate_series(1, " INT64_FORMAT ") as aid",
- //naccounts, (int64) naccounts * scale);
executeStatement(con, sql.data);
executeStatement(con, "commit");
--
2.43.0
From e47b52ddf23593dad9375ef5356fd41d0621ede3 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Mon, 10 Nov 2025 19:06:48 +0700
Subject: [PATCH v8 04/23] Adding server-side data generation via unnest
---
src/bin/pgbench/pgbench.c | 199 ++++++++++++++++++++++++++++++++++----
1 file changed, 182 insertions(+), 17 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 97895aa9edf..65d77cdefea 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -161,7 +161,7 @@ typedef struct socket_set
* some configurable parameters */
#define DEFAULT_INIT_STEPS "dtgvp" /* default -I setting */
-#define ALL_INIT_STEPS "dtgGvpf" /* all possible steps */
+#define ALL_INIT_STEPS "dtgGiIvpf" /* all possible steps */
#define LOG_STEP_SECONDS 5 /* seconds between log messages */
#define DEFAULT_NXACTS 10 /* default nxacts */
@@ -171,6 +171,12 @@ typedef struct socket_set
#define MIN_ZIPFIAN_PARAM 1.001 /* minimum parameter for zipfian */
#define MAX_ZIPFIAN_PARAM 1000.0 /* maximum parameter for zipfian */
+/* original single transaction server-side method */
+#define GEN_TYPE_INSERT_ORIGINAL 'G' /* use INSERT .. SELECT generate_series to generate data */
+/* 'one transaction per scale' server-side methods */
+#define GEN_TYPE_INSERT_SERIES 'i' /* use INSERT .. SELECT generate_series to generate data */
+#define GEN_TYPE_INSERT_UNNEST 'I' /* use INSERT .. SELECT unnest to generate data */
+
static int nxacts = 0; /* number of transactions per client */
static int duration = 0; /* duration in seconds */
static int64 end_time = 0; /* when to stop in micro seconds, under -T */
@@ -181,6 +187,11 @@ static int64 end_time = 0; /* when to stop in micro seconds, under -T */
*/
static int scale = 1;
+/*
+ *
+ */
+static char data_generation_type = '?';
+
/*
* fillfactor. for example, fillfactor = 90 will use only 90 percent
* space during inserts and leave 10 percent free.
@@ -914,7 +925,9 @@ usage(void)
" d: drop any existing pgbench tables\n"
" t: create the tables used by the standard pgbench scenario\n"
" g: generate data, client-side\n"
- " G: generate data, server-side\n"
+ " G: generate data, server-side in single transaction\n"
+ " i: server-side (multiple TXNs) INSERT .. SELECT generate_series\n"
+ " I: server-side (multiple TXNs) INSERT .. SELECT unnest\n"
" v: invoke VACUUM on the standard tables\n"
" p: create primary key indexes on the standard tables\n"
" f: create foreign keys between the standard tables\n"
@@ -5203,18 +5216,16 @@ initGenerateDataClientSide(PGconn *con)
}
/*
- * Fill the standard tables with some data generated on the server
- *
- * As already the case with the client-side data generation, the filler
- * column defaults to NULL in pgbench_branches and pgbench_tellers,
- * and is a blank-padded string in pgbench_accounts.
+ * Generating data via INSERT .. SELECT .. FROM generate_series
+ * whole dataset in single transaction
*/
static void
-initGenerateDataServerSide(PGconn *con)
+generateDataInsertSingleTXN(PGconn *con)
{
PQExpBufferData sql;
- fprintf(stderr, "generating data (server-side)...\n");
+ fprintf(stderr, "via INSERT .. SELECT generate_series... in single TXN\n");
+
/*
* we do all of this in one transaction to enable the backend's
@@ -5225,31 +5236,136 @@ initGenerateDataServerSide(PGconn *con)
/* truncate away any old data */
initTruncateTables(con);
+ initPQExpBuffer(&sql);
+
+ printfPQExpBuffer(&sql,
+ "insert into pgbench_branches(bid, bbalance) "
+ "select bid, 0 "
+ "from generate_series(1, %d)", scale * nbranches);
+ executeStatement(con, sql.data);
+
+ printfPQExpBuffer(&sql,
+ "insert into pgbench_tellers(tid, bid, tbalance) "
+ "select tid + 1, tid / %d + 1, 0 "
+ "from generate_series(0, %d) as tid",
+ ntellers, (scale * ntellers) - 1);
+ executeStatement(con, sql.data);
+
+ printfPQExpBuffer(&sql,
+ "insert into pgbench_accounts(aid, bid, abalance, "
+ "filler) "
+ "select aid + 1, aid / %d + 1, 0, '' "
+ "from generate_series(0, " INT64_FORMAT ") as aid",
+ naccounts, (int64) (scale * naccounts) - 1);
+ executeStatement(con, sql.data);
+
executeStatement(con, "commit");
+ termPQExpBuffer(&sql);
+}
+
+
+/*
+ * Generating data via INSERT .. SELECT .. FROM generate_series
+ * One transaction per 'scale'
+ */
+static void
+generateDataInsertSeries(PGconn *con)
+{
+ PQExpBufferData sql;
+
+ fprintf(stderr, "via INSERT .. SELECT generate_series... in multiple TXN(s)\n");
+
initPQExpBuffer(&sql);
- for (int i = 0; i < scale; i++) {
+ executeStatement(con, "begin");
+
+ /* truncate away any old data */
+ initTruncateTables(con);
+
+ executeStatement(con, "commit");
+
+ for (int i = 0; i < scale; i++)
+ {
executeStatement(con, "begin");
printfPQExpBuffer(&sql,
- "insert into pgbench_branches(bid,bbalance) "
- "select bid + 1, 0 "
- "from generate_series(%d, %d) as bid", i, i + 1);
+ "insert into pgbench_branches(bid, bbalance) "
+ "values(%d, 0)", i + 1);
executeStatement(con, sql.data);
printfPQExpBuffer(&sql,
- "insert into pgbench_tellers(tid,bid,tbalance) "
+ "insert into pgbench_tellers(tid, bid, tbalance) "
"select tid + 1, tid / %d + 1, 0 "
"from generate_series(%d, %d) as tid",
ntellers, i * ntellers, (i + 1) * ntellers - 1);
executeStatement(con, sql.data);
printfPQExpBuffer(&sql,
- "insert into pgbench_accounts(aid,bid,abalance,filler) "
+ "insert into pgbench_accounts(aid, bid, abalance, "
+ "filler) "
"select aid + 1, aid / %d + 1, 0, '' "
- "from generate_series(" INT64_FORMAT ", " INT64_FORMAT ") as aid",
- naccounts, (int64) i * naccounts, (int64) (i + 1) * naccounts - 1);
+ "from generate_series(" INT64_FORMAT ", "
+ INT64_FORMAT ") as aid",
+ naccounts, (int64) i * naccounts,
+ (int64) (i + 1) * naccounts - 1);
+ executeStatement(con, sql.data);
+
+ executeStatement(con, "commit");
+ }
+
+ termPQExpBuffer(&sql);
+}
+
+/*
+ * Generating data via INSERT .. SELECT .. FROM unnest
+ * One transaction per 'scale'
+ */
+static void
+generateDataInsertUnnest(PGconn *con)
+{
+ PQExpBufferData sql;
+
+ fprintf(stderr, "via INSERT .. SELECT unnest...\n");
+
+ initPQExpBuffer(&sql);
+
+ executeStatement(con, "begin");
+
+ /* truncate away any old data */
+ initTruncateTables(con);
+
+ executeStatement(con, "commit");
+
+ for (int s = 0; s < scale; s++)
+ {
+ executeStatement(con, "begin");
+
+ printfPQExpBuffer(&sql,
+ "insert into pgbench_branches(bid,bbalance) "
+ "values(%d, 0)", s + 1);
+ executeStatement(con, sql.data);
+
+ printfPQExpBuffer(&sql,
+ "insert into pgbench_tellers(tid, bid, tbalance) "
+ "select unnest(array_agg(s.i order by s.i)) as tid, "
+ "%d as bid, 0 as tbalance "
+ "from generate_series(%d, %d) as s(i)",
+ s + 1, s * ntellers + 1, (s + 1) * ntellers);
+ executeStatement(con, sql.data);
+
+ printfPQExpBuffer(&sql,
+ "with data as ("
+ " select generate_series(" INT64_FORMAT ", "
+ INT64_FORMAT ") as i) "
+ "insert into pgbench_accounts(aid, bid, "
+ "abalance, filler) "
+ "select unnest(aid), unnest(bid), 0 as abalance, "
+ "'' as filler "
+ "from (select array_agg(i+1) aid, "
+ "array_agg(i/%d + 1) bid from data)",
+ (int64) s * naccounts + 1,
+ (int64) (s + 1) * naccounts, naccounts);
executeStatement(con, sql.data);
executeStatement(con, "commit");
@@ -5258,6 +5374,32 @@ initGenerateDataServerSide(PGconn *con)
termPQExpBuffer(&sql);
}
+/*
+ * Fill the standard tables with some data generated on the server
+ *
+ * As already the case with the client-side data generation, the filler
+ * column defaults to NULL in pgbench_branches and pgbench_tellers,
+ * and is a blank-padded string in pgbench_accounts.
+ */
+static void
+initGenerateDataServerSide(PGconn *con)
+{
+ fprintf(stderr, "generating data (server-side) ");
+
+ switch (data_generation_type)
+ {
+ case GEN_TYPE_INSERT_ORIGINAL:
+ generateDataInsertSingleTXN(con);
+ break;
+ case GEN_TYPE_INSERT_SERIES:
+ generateDataInsertSeries(con);
+ break;
+ case GEN_TYPE_INSERT_UNNEST:
+ generateDataInsertUnnest(con);
+ break;
+ }
+}
+
/*
* Invoke vacuum on the standard tables
*/
@@ -5341,6 +5483,8 @@ initCreateFKeys(PGconn *con)
static void
checkInitSteps(const char *initialize_steps)
{
+ char data_init_type = 0;
+
if (initialize_steps[0] == '\0')
pg_fatal("no initialization steps specified");
@@ -5352,7 +5496,26 @@ checkInitSteps(const char *initialize_steps)
pg_log_error_detail("Allowed step characters are: \"" ALL_INIT_STEPS "\".");
exit(1);
}
+
+ switch (*step)
+ {
+ case 'G':
+ data_init_type++;
+ data_generation_type = *step;
+ break;
+ case 'i':
+ data_init_type++;
+ data_generation_type = *step;
+ break;
+ case 'I':
+ data_init_type++;
+ data_generation_type = *step;
+ break;
+ }
}
+
+ if (data_init_type > 1)
+ pg_log_error("WARNING! More than one type of server-side data generation is requested");
}
/*
@@ -5395,6 +5558,8 @@ runInitSteps(const char *initialize_steps)
initGenerateDataClientSide(con);
break;
case 'G':
+ case 'i':
+ case 'I':
op = "server-side generate";
initGenerateDataServerSide(con);
break;
--
2.43.0
From 5e1827b889b283f50299ce6ab1a73f9f55a4a84f Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Mon, 10 Nov 2025 20:00:56 +0700
Subject: [PATCH v8 05/23] Fixing typo in query
---
src/bin/pgbench/pgbench.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 65d77cdefea..03e37df4434 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -5241,7 +5241,8 @@ generateDataInsertSingleTXN(PGconn *con)
printfPQExpBuffer(&sql,
"insert into pgbench_branches(bid, bbalance) "
"select bid, 0 "
- "from generate_series(1, %d)", scale * nbranches);
+ "from generate_series(1, %d) as bid",
+ scale * nbranches);
executeStatement(con, sql.data);
printfPQExpBuffer(&sql,
--
2.43.0
From 7ca86521fda6929b8e0de3fc77dcbb8984009c88 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Tue, 11 Nov 2025 19:39:45 +0700
Subject: [PATCH v8 06/23] Adding support for COPY BINARY mode
---
src/bin/pgbench/pgbench.c | 393 ++++++++++++++++++++++++++++++++++++--
1 file changed, 381 insertions(+), 12 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 03e37df4434..71aa1d9479f 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -161,7 +161,7 @@ typedef struct socket_set
* some configurable parameters */
#define DEFAULT_INIT_STEPS "dtgvp" /* default -I setting */
-#define ALL_INIT_STEPS "dtgGiIvpf" /* all possible steps */
+#define ALL_INIT_STEPS "dtgCGiIvpf" /* all possible steps */
#define LOG_STEP_SECONDS 5 /* seconds between log messages */
#define DEFAULT_NXACTS 10 /* default nxacts */
@@ -176,6 +176,8 @@ typedef struct socket_set
/* 'one transaction per scale' server-side methods */
#define GEN_TYPE_INSERT_SERIES 'i' /* use INSERT .. SELECT generate_series to generate data */
#define GEN_TYPE_INSERT_UNNEST 'I' /* use INSERT .. SELECT unnest to generate data */
+#define GEN_TYPE_COPY_ORIGINAL 'g' /* use COPY .. FROM STDIN .. TEXT to generate data */
+#define GEN_TYPE_COPY_BINARY 'C' /* use COPY .. FROM STDIN .. BINARY to generate data */
static int nxacts = 0; /* number of transactions per client */
static int duration = 0; /* duration in seconds */
@@ -188,10 +190,17 @@ static int64 end_time = 0; /* when to stop in micro seconds, under -T */
static int scale = 1;
/*
- *
+ * mode of data generation to use
*/
static char data_generation_type = '?';
+/*
+ * COPY FROM BINARY execution buffer
+ */
+#define BIN_COPY_BUF_SIZE 102400 /* maximum buffer size for COPY FROM BINARY */
+static char *bin_copy_buffer = NULL; /* buffer for COPY FROM BINARY */
+static int32_t bin_copy_buffer_length = 0; /* current buffer size */
+
/*
* fillfactor. for example, fillfactor = 90 will use only 90 percent
* space during inserts and leave 10 percent free.
@@ -861,7 +870,8 @@ static int wait_on_socket_set(socket_set *sa, int64 usecs);
static bool socket_has_input(socket_set *sa, int fd, int idx);
/* callback used to build rows for COPY during data loading */
-typedef void (*initRowMethod) (PQExpBufferData *sql, int64 curr);
+typedef void (*initRowMethod) (PQExpBufferData *sql, int64 curr);
+typedef void (*initRowMethodBin) (PGconn *con, PGresult *res, int64_t curr, int32_t parent);
/* callback functions for our flex lexer */
static const PsqlScanCallbacks pgbench_callbacks = {
@@ -925,6 +935,7 @@ usage(void)
" d: drop any existing pgbench tables\n"
" t: create the tables used by the standard pgbench scenario\n"
" g: generate data, client-side\n"
+ " C: client-side (single TNX) COPY .. FROM STDIN .. BINARY\n"
" G: generate data, server-side in single transaction\n"
" i: server-side (multiple TXNs) INSERT .. SELECT generate_series\n"
" I: server-side (multiple TXNs) INSERT .. SELECT unnest\n"
@@ -5191,9 +5202,9 @@ initPopulateTable(PGconn *con, const char *table, int64 base,
* a blank-padded string in pgbench_accounts.
*/
static void
-initGenerateDataClientSide(PGconn *con)
+initGenerateDataClientSideText(PGconn *con)
{
- fprintf(stderr, "generating data (client-side)...\n");
+ fprintf(stderr, "TEXT mode...\n");
/*
* we do all of this in one transaction to enable the backend's
@@ -5209,12 +5220,373 @@ initGenerateDataClientSide(PGconn *con)
* already exist
*/
initPopulateTable(con, "pgbench_branches", nbranches, initBranch);
- initPopulateTable(con, "pgbench_tellers", ntellers, initTeller);
+ initPopulateTable(con, "pgbench_tellers", ntellers, initTeller);
initPopulateTable(con, "pgbench_accounts", naccounts, initAccount);
executeStatement(con, "commit");
}
+
+/*
+ * Dumps binary buffer to file (purely for debugging)
+ */
+static void
+dumpBufferToFile(char *filename)
+{
+ FILE *file_ptr;
+ size_t bytes_written;
+
+ file_ptr = fopen(filename, "wb");
+ if (file_ptr == NULL)
+ {
+ fprintf(stderr, "Error opening file %s\n", filename);
+ return; // EXIT_FAILURE;
+ }
+
+ bytes_written = fwrite(bin_copy_buffer, 1, bin_copy_buffer_length, file_ptr);
+
+ if (bytes_written != bin_copy_buffer_length)
+ {
+ fprintf(stderr, "Error writing to file or incomplete write\n");
+ fclose(file_ptr);
+ return; // EXIT_FAILURE;
+ }
+
+ fclose(file_ptr);
+}
+
+/*
+ * Save char data to buffer
+ */
+static void
+bufferCharData(char *src, int32_t len)
+{
+ memcpy((char *) bin_copy_buffer + bin_copy_buffer_length, (char *) src, len);
+ bin_copy_buffer_length += len;
+}
+
+/*
+ * Converts platform byte order into network byte order
+ * SPARC doesn't reqire that
+ */
+static void
+bufferData(void *src, int32_t len)
+{
+#ifdef __sparc__
+ bufferCharData(src, len);
+#else
+ if (len == 1)
+ bufferCharData(src, len);
+ else
+ for (int32_t i = 0; i < len; i++)
+ {
+ ((char *) bin_copy_buffer + bin_copy_buffer_length)[i] =
+ ((char *) src)[len - i - 1];
+ }
+
+ bin_copy_buffer_length += len;
+#endif
+}
+
+/*
+ * adds column counter
+ */
+static void
+addColumnCounter(int16_t n)
+{
+ bufferData((void *) &n, sizeof(n));
+}
+
+/*
+ * adds column with NULL value
+ */
+static void
+addNullColumn()
+{
+ int32_t null = -1;
+ bufferData((void *) &null, sizeof(null));
+}
+
+/*
+ * adds column with int8 value
+ */
+static void
+addInt8Column(int8_t value)
+{
+ int8_t data = value;
+ int32_t size = sizeof(data);
+ bufferData((void *) &size, sizeof(size));
+ bufferData((void *) &data, sizeof(data));
+}
+
+/*
+ * adds column with int16 value
+ */
+static void
+addInt16Column(int16_t value)
+{
+ int16_t data = value;
+ int32_t size = sizeof(data);
+ bufferData((void *) &size, sizeof(size));
+ bufferData((void *) &data, sizeof(data));
+}
+
+/*
+ * adds column with inti32 value
+ */
+static void
+addInt32Column(int32_t value)
+{
+ int32_t data = value;
+ int32_t size = sizeof(data);
+ bufferData((void *) &size, sizeof(size));
+ bufferData((void *) &data, sizeof(data));
+}
+
+/*
+ * adds column with inti64 value
+ */
+static void
+addInt64Column(int64_t value)
+{
+ int64_t data = value;
+ int32_t size = sizeof(data);
+ bufferData((void *) &size, sizeof(size));
+ bufferData((void *) &data, sizeof(data));
+}
+
+/*
+ * adds column with char value
+ */
+static void
+addCharColumn(char *value)
+{
+ int32_t size = strlen(value);
+ bufferData((void *) &size, sizeof(size));
+ bufferCharData(value, size);
+}
+
+/*
+ * Starts communication with server for COPY FROM BINARY statement
+ */
+static void
+sendBinaryCopyHeader(PGconn *con)
+{
+ char header[] = {'P','G','C','O','P','Y','\n','\377','\r','\n','\0',
+ '\0','\0','\0','\0',
+ '\0','\0','\0','\0' };
+
+ PQputCopyData(con, header, 19);
+}
+
+/*
+ * Finishes communication with server for COPY FROM BINARY statement
+ */
+static void
+sendBinaryCopyTrailer(PGconn *con)
+{
+ static char trailer[] = { 0xFF, 0xFF };
+
+ PQputCopyData(con, trailer, 2);
+}
+
+/*
+ * Flashes current buffer over network if needed
+ */
+static void
+flushBuffer(PGconn *con, PGresult *res, int16_t row_len)
+{
+ if (bin_copy_buffer_length + row_len > BIN_COPY_BUF_SIZE)
+ {
+ /* flush current buffer */
+ if (PQresultStatus(res) == PGRES_COPY_IN)
+ PQputCopyData(con, (char *) bin_copy_buffer, bin_copy_buffer_length);
+ bin_copy_buffer_length = 0;
+ }
+}
+
+/*
+ * Sends current branch row to buffer
+ */
+static void
+initBranchBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
+{
+ /*
+ * Each row has following extra bytes:
+ * - 2 bytes for number of columns
+ * - 4 bytes as length for each column
+ */
+ int16_t max_row_len = 35 + 2 + 4*3; /* max row size is 32 */
+
+ flushBuffer(con, res, max_row_len);
+
+ addColumnCounter(2);
+
+ addInt32Column(curr + 1);
+ addInt32Column(0);
+}
+
+/*
+ * Sends current teller row to buffer
+ */
+static void
+initTellerBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
+{
+ /*
+ * Each row has following extra bytes:
+ * - 2 bytes for number of columns
+ * - 4 bytes as length for each column
+ */
+ int16_t max_row_len = 40 + 2 + 4*4; /* max row size is 40 */
+
+ flushBuffer(con, res, max_row_len);
+
+ addColumnCounter(3);
+
+ addInt32Column(curr + 1);
+ addInt32Column(curr / parent + 1);
+ addInt32Column(0);
+}
+
+/*
+ * Sends current account row to buffer
+ */
+static void
+initAccountBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
+{
+ /*
+ * Each row has following extra bytes:
+ * - 2 bytes for number of columns
+ * - 4 bytes as length for each column
+ */
+ int16_t max_row_len = 250 + 2 + 4*4; /* max row size is 250 for int64 */
+
+ flushBuffer(con, res, max_row_len);
+
+ addColumnCounter(3);
+
+ if (scale <= SCALE_32BIT_THRESHOLD)
+ addInt32Column(curr + 1);
+ else
+ addInt64Column(curr);
+
+ addInt32Column(curr / parent + 1);
+ addInt32Column(0);
+}
+
+/*
+ * Universal wrapper for sending data in binary format
+ */
+static void
+initPopulateTableBinary(PGconn *con, char *table, char *columns,
+ int64_t base, initRowMethodBin init_row)
+{
+ int n;
+ PGresult *res;
+ char copy_statement[256];
+ const char *copy_statement_fmt = "copy %s (%s) from stdin (format binary)";
+ int64_t total = base * scale;
+
+ bin_copy_buffer_length = 0;
+
+ /* Use COPY with FREEZE on v14 and later for all ordinary tables */
+ if ((PQserverVersion(con) >= 140000) &&
+ get_table_relkind(con, table) == RELKIND_RELATION)
+ copy_statement_fmt = "copy %s (%s) from stdin with (format binary, freeze on)";
+
+ n = pg_snprintf(copy_statement, sizeof(copy_statement), copy_statement_fmt, table, columns);
+ if (n >= sizeof(copy_statement))
+ pg_fatal("invalid buffer size: must be at least %d characters long", n);
+ else if (n == -1)
+ pg_fatal("invalid format string");
+
+ res = PQexec(con, copy_statement);
+
+ if (PQresultStatus(res) != PGRES_COPY_IN)
+ pg_fatal("unexpected copy in result: %s", PQerrorMessage(con));
+ PQclear(res);
+
+
+ sendBinaryCopyHeader(con);
+
+ for (int64_t i = 0; i < total; i++)
+ {
+ init_row(con, res, i, base);
+ }
+
+ if (PQresultStatus(res) == PGRES_COPY_IN)
+ PQputCopyData(con, (char *) bin_copy_buffer, bin_copy_buffer_length);
+ else
+ fprintf(stderr, "Unexpected mode %d instead of %d\n", PQresultStatus(res), PGRES_COPY_IN);
+
+ sendBinaryCopyTrailer(con);
+
+ if (PQresultStatus(res) == PGRES_COPY_IN)
+ {
+ if (PQputCopyEnd(con, NULL) == 1) /* success */
+ {
+ res = PQgetResult(con);
+ if (PQresultStatus(res) != PGRES_COMMAND_OK)
+ fprintf(stderr, "Error: %s\n", PQerrorMessage(con));
+ PQclear(res);
+ }
+ else
+ fprintf(stderr, "Error: %s\n", PQerrorMessage(con));
+ }
+}
+
+/*
+ * Wrapper for binary data load
+ */
+static void
+initGenerateDataClientSideBinary(PGconn *con)
+{
+
+ fprintf(stderr, "BINARY mode...\n");
+
+ bin_copy_buffer = pg_malloc(BIN_COPY_BUF_SIZE);
+ bin_copy_buffer_length = 0;
+
+ /*
+ * we do all of this in one transaction to enable the backend's
+ * data-loading optimizations
+ */
+ executeStatement(con, "begin");
+
+ /* truncate away any old data */
+ initTruncateTables(con);
+
+ initPopulateTableBinary(con, "pgbench_branches", "bid, bbalance",
+ nbranches, initBranchBinary);
+ initPopulateTableBinary(con, "pgbench_tellers", "tid, bid, tbalance",
+ ntellers, initTellerBinary);
+ initPopulateTableBinary(con, "pgbench_accounts", "aid, bid, abalance",
+ naccounts, initAccountBinary);
+
+ executeStatement(con, "commit");
+
+ pg_free(bin_copy_buffer);
+}
+
+/*
+ * Fill the standard tables with some data generated and sent from the client.
+ */
+static void
+initGenerateDataClientSide(PGconn *con)
+{
+ fprintf(stderr, "generating data (client-side) in ");
+
+ switch (data_generation_type)
+ {
+ case GEN_TYPE_COPY_ORIGINAL:
+ initGenerateDataClientSideText(con);
+ break;
+ case GEN_TYPE_COPY_BINARY:
+ initGenerateDataClientSideBinary(con);
+ break;
+ }
+}
+
/*
* Generating data via INSERT .. SELECT .. FROM generate_series
* whole dataset in single transaction
@@ -5500,14 +5872,10 @@ checkInitSteps(const char *initialize_steps)
switch (*step)
{
+ case 'g':
+ case 'C':
case 'G':
- data_init_type++;
- data_generation_type = *step;
- break;
case 'i':
- data_init_type++;
- data_generation_type = *step;
- break;
case 'I':
data_init_type++;
data_generation_type = *step;
@@ -5555,6 +5923,7 @@ runInitSteps(const char *initialize_steps)
initCreateTables(con);
break;
case 'g':
+ case 'C':
op = "client-side generate";
initGenerateDataClientSide(con);
break;
--
2.43.0
From 4aa0ac05765edf6b5f0c13e18ac677287ce78206 Mon Sep 17 00:00:00 2001
From: Fujii Masao <[email protected]>
Date: Fri, 14 Nov 2025 22:40:39 +0900
Subject: [PATCH v8 07/23] pgbench: Fix assertion failure with multiple
\syncpipeline in pipeline mode.
Previously, when pgbench ran a custom script that triggered retriable errors
(e.g., deadlocks) followed by multiple \syncpipeline commands in pipeline mode,
the following assertion failure could occur:
Assertion failed: (res == ((void*)0)), function discardUntilSync, file pgbench.c, line 3594.
The issue was that discardUntilSync() assumed a pipeline sync result
(PGRES_PIPELINE_SYNC) would always be followed by either another sync result
or NULL. This assumption was incorrect: when multiple sync requests were sent,
a sync result could instead be followed by another result type. In such cases,
discardUntilSync() mishandled the results, leading to the assertion failure.
This commit fixes the issue by making discardUntilSync() correctly handle cases
where a pipeline sync result is followed by other result types. It now continues
discarding results until another pipeline sync followed by NULL is reached.
Backpatched to v17, where support for \syncpipeline command in pgbench was
introduced.
Author: Yugo Nagata <[email protected]>
Reviewed-by: Chao Li <[email protected]>
Reviewed-by: Fujii Masao <[email protected]>
Discussion: https://postgr.es/m/[email protected]
Backpatch-through: 17
---
src/bin/pgbench/pgbench.c | 39 ++++++++++++++++++++++++++++-----------
1 file changed, 28 insertions(+), 11 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index d8764ba6fe0..a425176ecdc 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -3563,14 +3563,18 @@ doRetry(CState *st, pg_time_usec_t *now)
}
/*
- * Read results and discard it until a sync point.
+ * Read and discard results until the last sync point.
*/
static int
discardUntilSync(CState *st)
{
bool received_sync = false;
- /* send a sync */
+ /*
+ * Send a Sync message to ensure at least one PGRES_PIPELINE_SYNC is
+ * received and to avoid an infinite loop, since all earlier ones may have
+ * already been received.
+ */
if (!PQpipelineSync(st->con))
{
pg_log_error("client %d aborted: failed to send a pipeline sync",
@@ -3578,29 +3582,42 @@ discardUntilSync(CState *st)
return 0;
}
- /* receive PGRES_PIPELINE_SYNC and null following it */
+ /*
+ * Continue reading results until the last sync point, i.e., until
+ * reaching null just after PGRES_PIPELINE_SYNC.
+ */
for (;;)
{
PGresult *res = PQgetResult(st->con);
+ if (PQstatus(st->con) == CONNECTION_BAD)
+ {
+ pg_log_error("client %d aborted while rolling back the transaction after an error; perhaps the backend died while processing",
+ st->id);
+ PQclear(res);
+ return 0;
+ }
+
if (PQresultStatus(res) == PGRES_PIPELINE_SYNC)
received_sync = true;
- else if (received_sync)
+ else if (received_sync && res == NULL)
{
- /*
- * PGRES_PIPELINE_SYNC must be followed by another
- * PGRES_PIPELINE_SYNC or NULL; otherwise, assert failure.
- */
- Assert(res == NULL);
-
/*
* Reset ongoing sync count to 0 since all PGRES_PIPELINE_SYNC
* results have been discarded.
*/
st->num_syncs = 0;
- PQclear(res);
break;
}
+ else
+ {
+ /*
+ * If a PGRES_PIPELINE_SYNC is followed by something other than
+ * PGRES_PIPELINE_SYNC or NULL, another PGRES_PIPELINE_SYNC will
+ * appear later. Reset received_sync to false to wait for it.
+ */
+ received_sync = false;
+ }
PQclear(res);
}
--
2.43.0
From 9c4f19055597e9adb25e65c2aa8bedf20a09e13d Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Fri, 21 Nov 2025 19:05:58 +0700
Subject: [PATCH v8 08/23] Setting empty string as default value in filler
column
---
src/bin/pgbench/pgbench.c | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 967f6ce6984..03b5e5c28f0 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -4985,26 +4985,26 @@ initCreateTables(PGconn *con)
static const struct ddlinfo DDLs[] = {
{
"pgbench_history",
- "tid int,bid int,aid int,delta int,mtime timestamp,filler char(22)",
- "tid int,bid int,aid bigint,delta int,mtime timestamp,filler char(22)",
+ "tid int,bid int,aid int,delta int,mtime timestamp,filler char(22) default ''",
+ "tid int,bid int,aid bigint,delta int,mtime timestamp,filler char(22) default ''",
0
},
{
"pgbench_tellers",
- "tid int not null,bid int,tbalance int,filler char(84)",
- "tid int not null,bid int,tbalance int,filler char(84)",
+ "tid int not null,bid int,tbalance int,filler char(84) default ''",
+ "tid int not null,bid int,tbalance int,filler char(84) default ''",
1
},
{
"pgbench_accounts",
- "aid int not null,bid int,abalance int,filler char(84)",
- "aid bigint not null,bid int,abalance int,filler char(84)",
+ "aid int not null,bid int,abalance int,filler char(84) default ''",
+ "aid bigint not null,bid int,abalance int,filler char(84) default ''",
1
},
{
"pgbench_branches",
- "bid int not null,bbalance int,filler char(88)",
- "bid int not null,bbalance int,filler char(88)",
+ "bid int not null,bbalance int,filler char(88) default ''",
+ "bid int not null,bbalance int,filler char(88) default ''",
1
}
};
--
2.43.0
From 2aabaa52dffdb78fbefaef95163881c15e18ef29 Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <[email protected]>
Date: Fri, 21 Nov 2025 15:03:11 +0200
Subject: [PATCH v8 09/23] Use strtoi64() in pgbench, replacing its open-coded
implementation
Makes the code a little simpler.
The old implementation accepted trailing whitespace, but that was
unnecessary. Firstly, its sibling function for parsing decimals,
strtodouble(), does not accept trailing whitespace. Secondly, none of
the callers can pass a string with trailing whitespace to it.
In the passing, check specifically for ERANGE before printing the "out
of range" error. On some systems, strtoul() and strtod() return EINVAL
on an empty or all-spaces string, and "invalid input syntax" is more
appropriate for that than "out of range". For the existing
strtodouble() function this is purely academical because it's never
called with errorOK==false, but let's be tidy. (Perhaps we should
remove the dead codepaths altogether, but I'll leave that for another
day.)
Reviewed-by: Chao Li <[email protected]>
Reviewed-by: Yuefei Shi <[email protected]>
Reviewed-by: Neil Chen <[email protected]>
Discussion: https://www.postgresql.org/message-id/[email protected]
---
src/bin/pgbench/pgbench.c | 83 +++++++++------------------------------
1 file changed, 19 insertions(+), 64 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index a425176ecdc..68774a59efd 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -982,13 +982,17 @@ usage(void)
progname, progname, PACKAGE_BUGREPORT, PACKAGE_NAME, PACKAGE_URL);
}
-/* return whether str matches "^\s*[-+]?[0-9]+$" */
+/*
+ * Return whether str matches "^\s*[-+]?[0-9]+$"
+ *
+ * This should agree with strtoint64() on what's accepted, ignoring overflows.
+ */
static bool
is_an_int(const char *str)
{
const char *ptr = str;
- /* skip leading spaces; cast is consistent with strtoint64 */
+ /* skip leading spaces */
while (*ptr && isspace((unsigned char) *ptr))
ptr++;
@@ -1012,9 +1016,6 @@ is_an_int(const char *str)
/*
* strtoint64 -- convert a string to 64-bit integer
*
- * This function is a slightly modified version of pg_strtoint64() from
- * src/backend/utils/adt/numutils.c.
- *
* The function returns whether the conversion worked, and if so
* "*result" is set to the result.
*
@@ -1023,71 +1024,25 @@ is_an_int(const char *str)
bool
strtoint64(const char *str, bool errorOK, int64 *result)
{
- const char *ptr = str;
- int64 tmp = 0;
- bool neg = false;
-
- /*
- * Do our own scan, rather than relying on sscanf which might be broken
- * for long long.
- *
- * As INT64_MIN can't be stored as a positive 64 bit integer, accumulate
- * value as a negative number.
- */
-
- /* skip leading spaces */
- while (*ptr && isspace((unsigned char) *ptr))
- ptr++;
-
- /* handle sign */
- if (*ptr == '-')
- {
- ptr++;
- neg = true;
- }
- else if (*ptr == '+')
- ptr++;
+ char *end;
- /* require at least one digit */
- if (unlikely(!isdigit((unsigned char) *ptr)))
- goto invalid_syntax;
+ errno = 0;
+ *result = strtoi64(str, &end, 10);
- /* process digits */
- while (*ptr && isdigit((unsigned char) *ptr))
+ if (unlikely(errno == ERANGE))
{
- int8 digit = (*ptr++ - '0');
-
- if (unlikely(pg_mul_s64_overflow(tmp, 10, &tmp)) ||
- unlikely(pg_sub_s64_overflow(tmp, digit, &tmp)))
- goto out_of_range;
+ if (!errorOK)
+ pg_log_error("value \"%s\" is out of range for type bigint", str);
+ return false;
}
- /* allow trailing whitespace, but not other trailing chars */
- while (*ptr != '\0' && isspace((unsigned char) *ptr))
- ptr++;
-
- if (unlikely(*ptr != '\0'))
- goto invalid_syntax;
-
- if (!neg)
+ if (unlikely(errno != 0 || end == str || *end != '\0'))
{
- if (unlikely(tmp == PG_INT64_MIN))
- goto out_of_range;
- tmp = -tmp;
+ if (!errorOK)
+ pg_log_error("invalid input syntax for type bigint: \"%s\"", str);
+ return false;
}
-
- *result = tmp;
return true;
-
-out_of_range:
- if (!errorOK)
- pg_log_error("value \"%s\" is out of range for type bigint", str);
- return false;
-
-invalid_syntax:
- if (!errorOK)
- pg_log_error("invalid input syntax for type bigint: \"%s\"", str);
- return false;
}
/* convert string to double, detecting overflows/underflows */
@@ -1099,14 +1054,14 @@ strtodouble(const char *str, bool errorOK, double *dv)
errno = 0;
*dv = strtod(str, &end);
- if (unlikely(errno != 0))
+ if (unlikely(errno == ERANGE))
{
if (!errorOK)
pg_log_error("value \"%s\" is out of range for type double", str);
return false;
}
- if (unlikely(end == str || *end != '\0'))
+ if (unlikely(errno != 0 || end == str || *end != '\0'))
{
if (!errorOK)
pg_log_error("invalid input syntax for type double: \"%s\"", str);
--
2.43.0
From dcb85d26f8132eaaf9d096e814b9bda49db7d478 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Fri, 21 Nov 2025 20:06:24 +0700
Subject: [PATCH v8 10/23] Switching COPY FROM BINARY ti run in multiple
transactions
---
src/bin/pgbench/pgbench.c | 27 ++++++++++++++++-----------
1 file changed, 16 insertions(+), 11 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 03b5e5c28f0..6b89007a63b 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -5496,20 +5496,20 @@ initAccountBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
*/
static void
initPopulateTableBinary(PGconn *con, char *table, char *columns,
- int64_t base, initRowMethodBin init_row)
+ int counter, int64_t base, initRowMethodBin init_row)
{
int n;
PGresult *res;
char copy_statement[256];
const char *copy_statement_fmt = "copy %s (%s) from stdin (format binary)";
- int64_t total = base * scale;
+ int64_t start = base * counter;
bin_copy_buffer_length = 0;
/* Use COPY with FREEZE on v14 and later for all ordinary tables */
if ((PQserverVersion(con) >= 140000) &&
get_table_relkind(con, table) == RELKIND_RELATION)
- copy_statement_fmt = "copy %s (%s) from stdin with (format binary, freeze on)";
+ copy_statement_fmt = "copy %s (%s) from stdin with (format binary)";
n = pg_snprintf(copy_statement, sizeof(copy_statement), copy_statement_fmt, table, columns);
if (n >= sizeof(copy_statement))
@@ -5526,7 +5526,7 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
sendBinaryCopyHeader(con);
- for (int64_t i = 0; i < total; i++)
+ for (int64_t i = start; i < start + base; i++)
{
init_row(con, res, i, base);
}
@@ -5573,15 +5573,20 @@ initGenerateDataClientSideBinary(PGconn *con)
/* truncate away any old data */
initTruncateTables(con);
- initPopulateTableBinary(con, "pgbench_branches", "bid, bbalance",
- nbranches, initBranchBinary);
- initPopulateTableBinary(con, "pgbench_tellers", "tid, bid, tbalance",
- ntellers, initTellerBinary);
- initPopulateTableBinary(con, "pgbench_accounts", "aid, bid, abalance",
- naccounts, initAccountBinary);
-
executeStatement(con, "commit");
+ for (int i = 0; i < scale; i++)
+ {
+ initPopulateTableBinary(con, "pgbench_branches", "bid, bbalance",
+ i, nbranches, initBranchBinary);
+ initPopulateTableBinary(con, "pgbench_tellers", "tid, bid, tbalance",
+ i, ntellers, initTellerBinary);
+ initPopulateTableBinary(con, "pgbench_accounts", "aid, bid, abalance",
+ i, naccounts, initAccountBinary);
+
+ executeStatement(con, "commit");
+ }
+
pg_free(bin_copy_buffer);
}
--
2.43.0
From b8e28881225234fd00b55235bc60fad2dc60b544 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Sat, 22 Nov 2025 17:06:00 +0700
Subject: [PATCH v8 11/23] Adding tests for new modes of data generation
---
src/bin/pgbench/pgbench.c | 21 ++++----
src/bin/pgbench/t/001_pgbench_with_server.pl | 52 +++++++++++++++++---
2 files changed, 56 insertions(+), 17 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 6b89007a63b..dd4e5d5e056 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -164,7 +164,7 @@ typedef struct socket_set
#define ALL_INIT_STEPS "dtgCGiIvpf" /* all possible steps */
#define LOG_STEP_SECONDS 5 /* seconds between log messages */
-#define DEFAULT_NXACTS 10 /* default nxacts */
+#define DEFAULT_NXACTS 10 /* default nxacts */
#define MIN_GAUSSIAN_PARAM 2.0 /* minimum parameter for gauss */
@@ -192,7 +192,7 @@ static int scale = 1;
/*
* mode of data generation to use
*/
-static char data_generation_type = '?';
+static char data_generation_type = GEN_TYPE_COPY_ORIGINAL;
/*
* COPY FROM BINARY execution buffer
@@ -4985,26 +4985,26 @@ initCreateTables(PGconn *con)
static const struct ddlinfo DDLs[] = {
{
"pgbench_history",
- "tid int,bid int,aid int,delta int,mtime timestamp,filler char(22) default ''",
- "tid int,bid int,aid bigint,delta int,mtime timestamp,filler char(22) default ''",
+ "tid int,bid int,aid int,delta int,mtime timestamp,filler char(22) default '?'",
+ "tid int,bid int,aid bigint,delta int,mtime timestamp,filler char(22) default '?'",
0
},
{
"pgbench_tellers",
- "tid int not null,bid int,tbalance int,filler char(84) default ''",
- "tid int not null,bid int,tbalance int,filler char(84) default ''",
+ "tid int not null,bid int,tbalance int,filler char(84)",
+ "tid int not null,bid int,tbalance int,filler char(84)",
1
},
{
"pgbench_accounts",
- "aid int not null,bid int,abalance int,filler char(84) default ''",
- "aid bigint not null,bid int,abalance int,filler char(84) default ''",
+ "aid int not null,bid int,abalance int,filler char(84) default '?'",
+ "aid bigint not null,bid int,abalance int,filler char(84) default '?'",
1
},
{
"pgbench_branches",
- "bid int not null,bbalance int,filler char(88) default ''",
- "bid int not null,bbalance int,filler char(88) default ''",
+ "bid int not null,bbalance int,filler char(88)",
+ "bid int not null,bbalance int,filler char(88)",
1
}
};
@@ -7837,6 +7837,7 @@ main(int argc, char **argv)
}
}
+ checkInitSteps(initialize_steps);
runInitSteps(initialize_steps);
exit(0);
}
diff --git a/src/bin/pgbench/t/001_pgbench_with_server.pl b/src/bin/pgbench/t/001_pgbench_with_server.pl
index 581e9af7907..a377048ead1 100644
--- a/src/bin/pgbench/t/001_pgbench_with_server.pl
+++ b/src/bin/pgbench/t/001_pgbench_with_server.pl
@@ -16,25 +16,30 @@ sub check_data_state
local $Test::Builder::Level = $Test::Builder::Level + 1;
my $node = shift;
my $type = shift;
+ my $sql_result;
- my $sql_result = $node->safe_psql('postgres',
- 'SELECT count(*) AS null_count FROM pgbench_accounts WHERE filler IS NULL LIMIT 10;'
- );
- is($sql_result, '0',
- "$type: filler column of pgbench_accounts has no NULL data");
$sql_result = $node->safe_psql('postgres',
'SELECT count(*) AS null_count FROM pgbench_branches WHERE filler IS NULL;'
);
is($sql_result, '1',
"$type: filler column of pgbench_branches has only NULL data");
+
$sql_result = $node->safe_psql('postgres',
'SELECT count(*) AS null_count FROM pgbench_tellers WHERE filler IS NULL;'
);
is($sql_result, '10',
"$type: filler column of pgbench_tellers has only NULL data");
+
+ $sql_result = $node->safe_psql('postgres',
+ 'SELECT count(*) AS null_count FROM pgbench_accounts WHERE filler IS NULL LIMIT 10;'
+ );
+ is($sql_result, '0',
+ "$type: filler column of pgbench_accounts has no NULL data");
+
$sql_result = $node->safe_psql('postgres',
'SELECT count(*) AS data_count FROM pgbench_history;');
- is($sql_result, '0', "$type: pgbench_history has no data");
+ is($sql_result, '0',
+ "$type: pgbench_history has no data");
}
# start a pgbench specific server
@@ -125,7 +130,7 @@ $node->pgbench(
'pgbench scale 1 initialization',);
# Check data state, after client-side data generation.
-check_data_state($node, 'client-side');
+check_data_state($node, 'client-side (default options)');
# Again, with all possible options
$node->pgbench(
@@ -143,6 +148,7 @@ $node->pgbench(
qr{done in \d+\.\d\d s }
],
'pgbench scale 1 initialization');
+check_data_state($node, 'client-side (all options)');
# Test interaction of --init-steps with legacy step-selection options
$node->pgbench(
@@ -164,6 +170,38 @@ $node->pgbench(
# Check data state, after server-side data generation.
check_data_state($node, 'server-side');
+# Test server-side generation with UNNEST
+$node->pgbench(
+ '--initialize --init-steps=dtI',
+ 0,
+ [qr{^$}],
+ [
+ qr{dropping old tables},
+ qr{creating tables},
+ qr{generating data \(server-side\)},
+ qr{done in \d+\.\d\d s }
+ ],
+ 'pgbench --init-steps server-side UNNEST');
+
+# Check data state, after server-side data generation.
+check_data_state($node, 'server-side (unnest)');
+
+# Test server-side generation with UNNEST
+$node->pgbench(
+ '--initialize --init-steps=dtC',
+ 0,
+ [qr{^$}],
+ [
+ qr{dropping old tables},
+ qr{creating tables},
+ qr{generating data \(client-side\)},
+ qr{done in \d+\.\d\d s }
+ ],
+ 'pgbench --init-steps client-side BINARY');
+
+# Check data state, after server-side data generation.
+check_data_state($node, 'client-side (binary)');
+
# Run all builtin scripts, for a few transactions each
$node->pgbench(
'--transactions=5 -Dfoo=bla --client=2 --protocol=simple --builtin=t'
--
2.43.0
From b3f2bce34232d299abc7644a8579f0ce49c8c9d6 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Sun, 23 Nov 2025 14:05:59 +0700
Subject: [PATCH v8 12/23] Fixing compiler warnings about unused procedures by
removing or commenting them out as they might be needed a bit later
---
src/bin/pgbench/pgbench.c | 31 +++++--------------------------
1 file changed, 5 insertions(+), 26 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 0a3ba21dcc9..682db61ff61 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -5201,7 +5201,7 @@ initGenerateDataClientSideText(PGconn *con)
/*
* Dumps binary buffer to file (purely for debugging)
- */
+ *
static void
dumpBufferToFile(char *filename)
{
@@ -5226,6 +5226,7 @@ dumpBufferToFile(char *filename)
fclose(file_ptr);
}
+ */
/*
* Save char data to buffer
@@ -5271,37 +5272,14 @@ addColumnCounter(int16_t n)
/*
* adds column with NULL value
- */
+ *
static void
addNullColumn()
{
int32_t null = -1;
bufferData((void *) &null, sizeof(null));
}
-
-/*
- * adds column with int8 value
*/
-static void
-addInt8Column(int8_t value)
-{
- int8_t data = value;
- int32_t size = sizeof(data);
- bufferData((void *) &size, sizeof(size));
- bufferData((void *) &data, sizeof(data));
-}
-
-/*
- * adds column with int16 value
- */
-static void
-addInt16Column(int16_t value)
-{
- int16_t data = value;
- int32_t size = sizeof(data);
- bufferData((void *) &size, sizeof(size));
- bufferData((void *) &data, sizeof(data));
-}
/*
* adds column with inti32 value
@@ -5329,7 +5307,7 @@ addInt64Column(int64_t value)
/*
* adds column with char value
- */
+ *
static void
addCharColumn(char *value)
{
@@ -5337,6 +5315,7 @@ addCharColumn(char *value)
bufferData((void *) &size, sizeof(size));
bufferCharData(value, size);
}
+ */
/*
* Starts communication with server for COPY FROM BINARY statement
--
2.43.0
From 9ab7fe302ba7de40593e7cba8ea1ca3b876c1ea5 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Thu, 29 Jan 2026 19:22:16 +0700
Subject: [PATCH v8 13/23] Moving PQclear call to the end of procedure to avoid
access to previously released memory
---
src/bin/pgbench/pgbench.c | 27 +++++++++++++++++----------
1 file changed, 17 insertions(+), 10 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 682db61ff61..3687b65871e 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -5363,11 +5363,11 @@ static void
initBranchBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
{
/*
- * Each row has following extra bytes:
+ * Each row of branch table has following extra bytes:
* - 2 bytes for number of columns
- * - 4 bytes as length for each column
+ * - 4 bytes as length for each of table's 3 columns
*/
- int16_t max_row_len = 35 + 2 + 4*3; /* max row size is 32 */
+ int16_t max_row_len = 35 + 2 + 4*3; /* max row size is 35 */
flushBuffer(con, res, max_row_len);
@@ -5384,9 +5384,9 @@ static void
initTellerBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
{
/*
- * Each row has following extra bytes:
+ * Each row of tellers table has following extra bytes:
* - 2 bytes for number of columns
- * - 4 bytes as length for each column
+ * - 4 bytes as length for each of table's 4 columns
*/
int16_t max_row_len = 40 + 2 + 4*4; /* max row size is 40 */
@@ -5406,9 +5406,9 @@ static void
initAccountBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
{
/*
- * Each row has following extra bytes:
+ * Each row of accounts table has following extra bytes:
* - 2 bytes for number of columns
- * - 4 bytes as length for each column
+ * - 4 bytes as length for each of table's 4 columns
*/
int16_t max_row_len = 250 + 2 + 4*4; /* max row size is 250 for int64 */
@@ -5455,11 +5455,11 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
if (PQresultStatus(res) != PGRES_COPY_IN)
pg_fatal("unexpected copy in result: %s", PQerrorMessage(con));
- PQclear(res);
sendBinaryCopyHeader(con);
+
for (int64_t i = start; i < start + base; i++)
{
init_row(con, res, i, base);
@@ -5470,8 +5470,10 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
else
fprintf(stderr, "Unexpected mode %d instead of %d\n", PQresultStatus(res), PGRES_COPY_IN);
+
sendBinaryCopyTrailer(con);
+
if (PQresultStatus(res) == PGRES_COPY_IN)
{
if (PQputCopyEnd(con, NULL) == 1) /* success */
@@ -5484,6 +5486,8 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
else
fprintf(stderr, "Error: %s\n", PQerrorMessage(con));
}
+
+ PQclear(res);
}
/*
@@ -5499,8 +5503,9 @@ initGenerateDataClientSideBinary(PGconn *con)
bin_copy_buffer_length = 0;
/*
- * we do all of this in one transaction to enable the backend's
- * data-loading optimizations
+ * we do all of this in multiple transactions
+ * to minimize load on DB server and perhaps
+ * in future allow load in parallel sessions
*/
executeStatement(con, "begin");
@@ -5511,6 +5516,8 @@ initGenerateDataClientSideBinary(PGconn *con)
for (int i = 0; i < scale; i++)
{
+ executeStatement(con, "begin");
+
initPopulateTableBinary(con, "pgbench_branches", "bid, bbalance",
i, nbranches, initBranchBinary);
initPopulateTableBinary(con, "pgbench_tellers", "tid, bid, tbalance",
--
2.43.0
From 3d7b2c53b3fa45b8848369bfbc49a0c7e5854304 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Thu, 29 Jan 2026 19:47:26 +0700
Subject: [PATCH v8 14/23] Fixing error about freeing memory twice
---
src/bin/pgbench/pgbench.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 3687b65871e..6f42438d970 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -5462,6 +5462,7 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
for (int64_t i = start; i < start + base; i++)
{
+ res = PQgetResult(con);
init_row(con, res, i, base);
}
@@ -5473,7 +5474,7 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
sendBinaryCopyTrailer(con);
-
+ res = PQgetResult(con);
if (PQresultStatus(res) == PGRES_COPY_IN)
{
if (PQputCopyEnd(con, NULL) == 1) /* success */
@@ -5481,7 +5482,6 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
res = PQgetResult(con);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
fprintf(stderr, "Error: %s\n", PQerrorMessage(con));
- PQclear(res);
}
else
fprintf(stderr, "Error: %s\n", PQerrorMessage(con));
--
2.43.0
From a71c90f556ba143335d9631ebbdf96b7120c6d1d Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Thu, 29 Jan 2026 20:24:54 +0700
Subject: [PATCH v8 15/23] Fixing memory leak during data flush in COPY BINARY
test
---
src/bin/pgbench/pgbench.c | 26 ++++++++++++++++----------
1 file changed, 16 insertions(+), 10 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 6f42438d970..8eca537b718 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -871,7 +871,7 @@ static bool socket_has_input(socket_set *sa, int fd, int idx);
/* callback used to build rows for COPY during data loading */
typedef void (*initRowMethod) (PQExpBufferData *sql, int64 curr);
-typedef void (*initRowMethodBin) (PGconn *con, PGresult *res, int64_t curr, int32_t parent);
+typedef void (*initRowMethodBin) (PGconn *con, int64_t curr, int32_t parent);
/* callback functions for our flex lexer */
static const PsqlScanCallbacks pgbench_callbacks = {
@@ -5345,13 +5345,18 @@ sendBinaryCopyTrailer(PGconn *con)
* Flashes current buffer over network if needed
*/
static void
-flushBuffer(PGconn *con, PGresult *res, int16_t row_len)
+flushBuffer(PGconn *con, int16_t row_len)
{
+ PGresult *res;
+
if (bin_copy_buffer_length + row_len > BIN_COPY_BUF_SIZE)
{
+ res = PQgetResult(con);
/* flush current buffer */
if (PQresultStatus(res) == PGRES_COPY_IN)
PQputCopyData(con, (char *) bin_copy_buffer, bin_copy_buffer_length);
+ else
+ pg_fatal("It is NOT a COPY command that is currently running");
bin_copy_buffer_length = 0;
}
}
@@ -5360,7 +5365,7 @@ flushBuffer(PGconn *con, PGresult *res, int16_t row_len)
* Sends current branch row to buffer
*/
static void
-initBranchBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
+initBranchBinary(PGconn *con, int64_t curr, int32_t parent)
{
/*
* Each row of branch table has following extra bytes:
@@ -5369,7 +5374,7 @@ initBranchBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
*/
int16_t max_row_len = 35 + 2 + 4*3; /* max row size is 35 */
- flushBuffer(con, res, max_row_len);
+ flushBuffer(con, max_row_len);
addColumnCounter(2);
@@ -5381,7 +5386,7 @@ initBranchBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
* Sends current teller row to buffer
*/
static void
-initTellerBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
+initTellerBinary(PGconn *con, int64_t curr, int32_t parent)
{
/*
* Each row of tellers table has following extra bytes:
@@ -5390,7 +5395,7 @@ initTellerBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
*/
int16_t max_row_len = 40 + 2 + 4*4; /* max row size is 40 */
- flushBuffer(con, res, max_row_len);
+ flushBuffer(con, max_row_len);
addColumnCounter(3);
@@ -5403,7 +5408,7 @@ initTellerBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
* Sends current account row to buffer
*/
static void
-initAccountBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
+initAccountBinary(PGconn *con, int64_t curr, int32_t parent)
{
/*
* Each row of accounts table has following extra bytes:
@@ -5412,7 +5417,7 @@ initAccountBinary(PGconn *con, PGresult *res, int64_t curr, int32_t parent)
*/
int16_t max_row_len = 250 + 2 + 4*4; /* max row size is 250 for int64 */
- flushBuffer(con, res, max_row_len);
+ flushBuffer(con, max_row_len);
addColumnCounter(3);
@@ -5462,10 +5467,10 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
for (int64_t i = start; i < start + base; i++)
{
- res = PQgetResult(con);
- init_row(con, res, i, base);
+ init_row(con, i, base);
}
+ res = PQgetResult(con);
if (PQresultStatus(res) == PGRES_COPY_IN)
PQputCopyData(con, (char *) bin_copy_buffer, bin_copy_buffer_length);
else
@@ -5474,6 +5479,7 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
sendBinaryCopyTrailer(con);
+
res = PQgetResult(con);
if (PQresultStatus(res) == PGRES_COPY_IN)
{
--
2.43.0
From a5933f40be0f3d408bb920f296fb5119c3ec28f4 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Fri, 30 Jan 2026 13:59:36 +0700
Subject: [PATCH v8 16/23] Fixing memory leak shown by valgrind
---
src/bin/pgbench/pgbench.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 8eca537b718..466d9023d72 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -5357,6 +5357,8 @@ flushBuffer(PGconn *con, int16_t row_len)
PQputCopyData(con, (char *) bin_copy_buffer, bin_copy_buffer_length);
else
pg_fatal("It is NOT a COPY command that is currently running");
+
+ PQclear(res);
bin_copy_buffer_length = 0;
}
}
@@ -5460,6 +5462,7 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
if (PQresultStatus(res) != PGRES_COPY_IN)
pg_fatal("unexpected copy in result: %s", PQerrorMessage(con));
+ PQclear(res);
sendBinaryCopyHeader(con);
@@ -5475,6 +5478,7 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
PQputCopyData(con, (char *) bin_copy_buffer, bin_copy_buffer_length);
else
fprintf(stderr, "Unexpected mode %d instead of %d\n", PQresultStatus(res), PGRES_COPY_IN);
+ PQclear(res);
sendBinaryCopyTrailer(con);
@@ -5485,6 +5489,7 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
{
if (PQputCopyEnd(con, NULL) == 1) /* success */
{
+ PQclear(res);
res = PQgetResult(con);
if (PQresultStatus(res) != PGRES_COMMAND_OK)
fprintf(stderr, "Error: %s\n", PQerrorMessage(con));
@@ -5492,7 +5497,6 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
else
fprintf(stderr, "Error: %s\n", PQerrorMessage(con));
}
-
PQclear(res);
}
--
2.43.0
From 6a3036f898b050d5e71e70eaceccd63003aa6444 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Fri, 6 Feb 2026 13:53:07 +0700
Subject: [PATCH v8 17/23] Adding ability to switch data init between single
and multiple transactions
---
src/bin/pgbench/pgbench.c | 192 +++++++++----------
src/bin/pgbench/t/001_pgbench_with_server.pl | 93 ++++++++-
2 files changed, 173 insertions(+), 112 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 466d9023d72..af71e358e71 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -159,9 +159,8 @@ typedef struct socket_set
/********************************************************************
* some configurable parameters */
-
-#define DEFAULT_INIT_STEPS "dtgvp" /* default -I setting */
-#define ALL_INIT_STEPS "dtgCGiIvpf" /* all possible steps */
+#define DEFAULT_INIT_STEPS "dtgvp" /* default -I setting */
+#define ALL_INIT_STEPS "dtgMScGUvpf" /* all possible steps */
#define LOG_STEP_SECONDS 5 /* seconds between log messages */
#define DEFAULT_NXACTS 10 /* default nxacts */
@@ -171,14 +170,17 @@ typedef struct socket_set
#define MIN_ZIPFIAN_PARAM 1.001 /* minimum parameter for zipfian */
#define MAX_ZIPFIAN_PARAM 1000.0 /* maximum parameter for zipfian */
-/* original single transaction server-side method */
-#define GEN_TYPE_INSERT_ORIGINAL 'G' /* use INSERT .. SELECT generate_series to generate data */
-/* 'one transaction per scale' server-side methods */
-#define GEN_TYPE_INSERT_SERIES 'i' /* use INSERT .. SELECT generate_series to generate data */
-#define GEN_TYPE_INSERT_UNNEST 'I' /* use INSERT .. SELECT unnest to generate data */
-#define GEN_TYPE_COPY_ORIGINAL 'g' /* use COPY .. FROM STDIN .. TEXT to generate data */
-#define GEN_TYPE_COPY_BINARY 'C' /* use COPY .. FROM STDIN .. BINARY to generate data */
-
+/* server-side methods to generate data */
+#define INIT_STEP_GEN_TYPE_INSERT_SERIES 'G' /* use INSERT .. SELECT generate_series to generate data */
+#define INIT_STEP_GEN_TYPE_INSERT_UNNEST 'U' /* use INSERT .. SELECT unnest to generate data */
+/* client-side methods to generate data */
+#define INIT_STEP_GEN_TYPE_COPY_TEXT 'g' /* use COPY .. FROM STDIN .. TEXT to generate data */
+#define INIT_STEP_GEN_TYPE_COPY_BINARY 'c' /* use COPY .. FROM STDIN .. BINARY to generate data */
+/* data init pseudo steps */
+#define INIT_STEP_GEN_TYPE_SINGLE_XACT 'S' /* switch to init data as single transaction */
+#define INIT_STEP_GEN_TYPE_MULTI_XACT 'M' /* switch to init data as multiple transactions */
+
+static bool multi_xact = false; /* init data type (as single or multiple transactions) */
static int nxacts = 0; /* number of transactions per client */
static int duration = 0; /* duration in seconds */
static int64 end_time = 0; /* when to stop in micro seconds, under -T */
@@ -192,7 +194,7 @@ static int scale = 1;
/*
* mode of data generation to use
*/
-static char data_generation_type = GEN_TYPE_COPY_ORIGINAL;
+static char data_generation_type = INIT_STEP_GEN_TYPE_COPY_TEXT;
/*
* COPY FROM BINARY execution buffer
@@ -934,11 +936,13 @@ usage(void)
" run selected initialization steps, in the specified order\n"
" d: drop any existing pgbench tables\n"
" t: create the tables used by the standard pgbench scenario\n"
- " g: generate data, client-side\n"
- " C: client-side (single TNX) COPY .. FROM STDIN .. BINARY\n"
- " G: generate data, server-side in single transaction\n"
- " i: server-side (multiple TXNs) INSERT .. SELECT generate_series\n"
- " I: server-side (multiple TXNs) INSERT .. SELECT unnest\n"
+ " to generate data, client-side:\n"
+ " g: COPY .. FROM STDIN .. TEXT\n"
+ " c: COPY .. FROM STDIN .. BINARY\n"
+ " to generate data, server-side:\n"
+ " G: INSERT .. SELECT generate_series\n"
+ " U: INSERT .. SELECT unnest\n"
+ " M: use multiple transactions to initialize data\n"
" v: invoke VACUUM on the standard tables\n"
" p: create primary key indexes on the standard tables\n"
" f: create foreign keys between the standard tables\n"
@@ -5049,8 +5053,8 @@ initAccount(PQExpBufferData *sql, int64 curr)
}
static void
-initPopulateTable(PGconn *con, const char *table, int64 base,
- initRowMethod init_row)
+initPopulateTableText(PGconn *con, const char *table, int64 base,
+ initRowMethod init_row)
{
int n;
int64 k;
@@ -5178,6 +5182,9 @@ initGenerateDataClientSideText(PGconn *con)
{
fprintf(stderr, "TEXT mode...\n");
+ if (multi_xact)
+ fprintf(stderr, "WARNING! Multiple transactions are not supported in this mode\n");
+
/*
* we do all of this in one transaction to enable the backend's
* data-loading optimizations
@@ -5191,9 +5198,9 @@ initGenerateDataClientSideText(PGconn *con)
* fill branches, tellers, accounts in that order in case foreign keys
* already exist
*/
- initPopulateTable(con, "pgbench_branches", nbranches, initBranch);
- initPopulateTable(con, "pgbench_tellers", ntellers, initTeller);
- initPopulateTable(con, "pgbench_accounts", naccounts, initAccount);
+ initPopulateTableText(con, "pgbench_branches", nbranches, initBranch);
+ initPopulateTableText(con, "pgbench_tellers", ntellers, initTeller);
+ initPopulateTableText(con, "pgbench_accounts", naccounts, initAccount);
executeStatement(con, "commit");
}
@@ -5450,7 +5457,8 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
/* Use COPY with FREEZE on v14 and later for all ordinary tables */
if ((PQserverVersion(con) >= 140000) &&
get_table_relkind(con, table) == RELKIND_RELATION)
- copy_statement_fmt = "copy %s (%s) from stdin with (format binary)";
+ if (!multi_xact)
+ copy_statement_fmt = "copy %s (%s) from stdin with (format binary, freeze on)";
n = pg_snprintf(copy_statement, sizeof(copy_statement), copy_statement_fmt, table, columns);
if (n >= sizeof(copy_statement))
@@ -5522,11 +5530,13 @@ initGenerateDataClientSideBinary(PGconn *con)
/* truncate away any old data */
initTruncateTables(con);
- executeStatement(con, "commit");
+ if (multi_xact)
+ executeStatement(con, "commit");
for (int i = 0; i < scale; i++)
{
- executeStatement(con, "begin");
+ if (multi_xact)
+ executeStatement(con, "begin");
initPopulateTableBinary(con, "pgbench_branches", "bid, bbalance",
i, nbranches, initBranchBinary);
@@ -5535,9 +5545,13 @@ initGenerateDataClientSideBinary(PGconn *con)
initPopulateTableBinary(con, "pgbench_accounts", "aid, bid, abalance",
i, naccounts, initAccountBinary);
- executeStatement(con, "commit");
+ if (multi_xact)
+ executeStatement(con, "commit");
}
+ if (!multi_xact)
+ executeStatement(con, "commit");
+
pg_free(bin_copy_buffer);
}
@@ -5547,14 +5561,15 @@ initGenerateDataClientSideBinary(PGconn *con)
static void
initGenerateDataClientSide(PGconn *con)
{
- fprintf(stderr, "generating data (client-side) in ");
+ fprintf(stderr, "generating data (client-side as %s transaction%s) in ",
+ multi_xact ? "multiple" : "single", multi_xact ? "s" : "");
switch (data_generation_type)
{
- case GEN_TYPE_COPY_ORIGINAL:
+ case INIT_STEP_GEN_TYPE_COPY_TEXT:
initGenerateDataClientSideText(con);
break;
- case GEN_TYPE_COPY_BINARY:
+ case INIT_STEP_GEN_TYPE_COPY_BINARY:
initGenerateDataClientSideBinary(con);
break;
}
@@ -5562,58 +5577,7 @@ initGenerateDataClientSide(PGconn *con)
/*
* Generating data via INSERT .. SELECT .. FROM generate_series
- * whole dataset in single transaction
- */
-static void
-generateDataInsertSingleTXN(PGconn *con)
-{
- PQExpBufferData sql;
-
- fprintf(stderr, "via INSERT .. SELECT generate_series... in single TXN\n");
-
-
- /*
- * we do all of this in one transaction to enable the backend's
- * data-loading optimizations
- */
- executeStatement(con, "begin");
-
- /* truncate away any old data */
- initTruncateTables(con);
-
- initPQExpBuffer(&sql);
-
- printfPQExpBuffer(&sql,
- "insert into pgbench_branches(bid, bbalance) "
- "select bid, 0 "
- "from generate_series(1, %d) as bid",
- scale * nbranches);
- executeStatement(con, sql.data);
-
- printfPQExpBuffer(&sql,
- "insert into pgbench_tellers(tid, bid, tbalance) "
- "select tid + 1, tid / %d + 1, 0 "
- "from generate_series(0, %d) as tid",
- ntellers, (scale * ntellers) - 1);
- executeStatement(con, sql.data);
-
- printfPQExpBuffer(&sql,
- "insert into pgbench_accounts(aid, bid, abalance, "
- "filler) "
- "select aid + 1, aid / %d + 1, 0, '' "
- "from generate_series(0, " INT64_FORMAT ") as aid",
- naccounts, (int64) (scale * naccounts) - 1);
- executeStatement(con, sql.data);
-
- executeStatement(con, "commit");
-
- termPQExpBuffer(&sql);
-}
-
-
-/*
- * Generating data via INSERT .. SELECT .. FROM generate_series
- * One transaction per 'scale'
+ * Possibly as "One transaction per scale" in multi-transaction mode
*/
static void
generateDataInsertSeries(PGconn *con)
@@ -5629,11 +5593,13 @@ generateDataInsertSeries(PGconn *con)
/* truncate away any old data */
initTruncateTables(con);
- executeStatement(con, "commit");
+ if (multi_xact)
+ executeStatement(con, "commit");
for (int i = 0; i < scale; i++)
{
- executeStatement(con, "begin");
+ if (multi_xact)
+ executeStatement(con, "begin");
printfPQExpBuffer(&sql,
"insert into pgbench_branches(bid, bbalance) "
@@ -5657,15 +5623,19 @@ generateDataInsertSeries(PGconn *con)
(int64) (i + 1) * naccounts - 1);
executeStatement(con, sql.data);
- executeStatement(con, "commit");
+ if (multi_xact)
+ executeStatement(con, "commit");
}
+ if (!multi_xact)
+ executeStatement(con, "commit");
+
termPQExpBuffer(&sql);
}
/*
* Generating data via INSERT .. SELECT .. FROM unnest
- * One transaction per 'scale'
+ * Possibly as "One transaction per scale" in multi-tansaction mode
*/
static void
generateDataInsertUnnest(PGconn *con)
@@ -5681,11 +5651,13 @@ generateDataInsertUnnest(PGconn *con)
/* truncate away any old data */
initTruncateTables(con);
- executeStatement(con, "commit");
+ if (multi_xact)
+ executeStatement(con, "commit");
for (int s = 0; s < scale; s++)
{
- executeStatement(con, "begin");
+ if (multi_xact)
+ executeStatement(con, "begin");
printfPQExpBuffer(&sql,
"insert into pgbench_branches(bid,bbalance) "
@@ -5714,14 +5686,18 @@ generateDataInsertUnnest(PGconn *con)
(int64) (s + 1) * naccounts, naccounts);
executeStatement(con, sql.data);
- executeStatement(con, "commit");
+ if (multi_xact)
+ executeStatement(con, "commit");
}
+ if (!multi_xact)
+ executeStatement(con, "commit");
+
termPQExpBuffer(&sql);
}
/*
- * Fill the standard tables with some data generated on the server
+ * Fill the standard tables with some data generated on the server side
*
* As already the case with the client-side data generation, the filler
* column defaults to NULL in pgbench_branches and pgbench_tellers,
@@ -5730,17 +5706,15 @@ generateDataInsertUnnest(PGconn *con)
static void
initGenerateDataServerSide(PGconn *con)
{
- fprintf(stderr, "generating data (server-side) ");
+ fprintf(stderr, "generating data (server-side as %s transaction%s) ",
+ multi_xact ? "multiple" : "single", multi_xact ? "s" : "");
switch (data_generation_type)
{
- case GEN_TYPE_INSERT_ORIGINAL:
- generateDataInsertSingleTXN(con);
- break;
- case GEN_TYPE_INSERT_SERIES:
+ case INIT_STEP_GEN_TYPE_INSERT_SERIES:
generateDataInsertSeries(con);
break;
- case GEN_TYPE_INSERT_UNNEST:
+ case INIT_STEP_GEN_TYPE_INSERT_UNNEST:
generateDataInsertUnnest(con);
break;
}
@@ -5845,19 +5819,20 @@ checkInitSteps(const char *initialize_steps)
switch (*step)
{
- case 'g':
- case 'C':
- case 'G':
- case 'i':
- case 'I':
+ case INIT_STEP_GEN_TYPE_COPY_TEXT:
+ case INIT_STEP_GEN_TYPE_COPY_BINARY:
+ case INIT_STEP_GEN_TYPE_INSERT_SERIES:
+ case INIT_STEP_GEN_TYPE_INSERT_UNNEST:
data_init_type++;
data_generation_type = *step;
break;
}
}
+ if (data_init_type == 0)
+ pg_log_error("WARNING! No data generation type is provided");
if (data_init_type > 1)
- pg_log_error("WARNING! More than one type of server-side data generation is requested");
+ pg_log_error("WARNING! More than one type of data initialization is requested");
}
/*
@@ -5895,17 +5870,22 @@ runInitSteps(const char *initialize_steps)
op = "create tables";
initCreateTables(con);
break;
- case 'g':
- case 'C':
+ case INIT_STEP_GEN_TYPE_COPY_TEXT:
+ case INIT_STEP_GEN_TYPE_COPY_BINARY:
op = "client-side generate";
initGenerateDataClientSide(con);
break;
- case 'G':
- case 'i':
- case 'I':
+ case INIT_STEP_GEN_TYPE_INSERT_SERIES:
+ case INIT_STEP_GEN_TYPE_INSERT_UNNEST:
op = "server-side generate";
initGenerateDataServerSide(con);
break;
+ case INIT_STEP_GEN_TYPE_SINGLE_XACT:
+ multi_xact = false;
+ break;
+ case INIT_STEP_GEN_TYPE_MULTI_XACT:
+ multi_xact = true;
+ break;
case 'v':
op = "vacuum";
initVacuum(con);
diff --git a/src/bin/pgbench/t/001_pgbench_with_server.pl b/src/bin/pgbench/t/001_pgbench_with_server.pl
index a377048ead1..acb1e31d3e8 100644
--- a/src/bin/pgbench/t/001_pgbench_with_server.pl
+++ b/src/bin/pgbench/t/001_pgbench_with_server.pl
@@ -117,6 +117,7 @@ $node->pgbench(
[qr{Perhaps you need to do initialization}],
'run without init');
+
# Initialize pgbench tables scale 1
$node->pgbench(
'-i', 0,
@@ -160,7 +161,7 @@ $node->pgbench(
qr{creating tables},
qr{creating 3 partitions},
qr{creating primary keys},
- qr{generating data \(server-side\)},
+ qr{generating data \(server-side as single transaction\)},
qr{creating foreign keys},
qr{(?!vacuuming)}, # no vacuum
qr{done in \d+\.\d\d s }
@@ -170,15 +171,16 @@ $node->pgbench(
# Check data state, after server-side data generation.
check_data_state($node, 'server-side');
+
# Test server-side generation with UNNEST
$node->pgbench(
- '--initialize --init-steps=dtI',
+ '--initialize --init-steps=dtU',
0,
[qr{^$}],
[
qr{dropping old tables},
qr{creating tables},
- qr{generating data \(server-side\)},
+ qr{generating data \(server-side as single transaction\)},
qr{done in \d+\.\d\d s }
],
'pgbench --init-steps server-side UNNEST');
@@ -186,15 +188,48 @@ $node->pgbench(
# Check data state, after server-side data generation.
check_data_state($node, 'server-side (unnest)');
-# Test server-side generation with UNNEST
+
+# Test server-side generation with COPY TEXT
+$node->pgbench(
+ '--initialize --init-steps=dtg',
+ 0,
+ [qr{^$}],
+ [
+ qr{dropping old tables},
+ qr{creating tables},
+ qr{generating data \(client-side as single transaction},
+ qr{done in \d+\.\d\d s }
+ ],
+ 'pgbench --init-steps client-side TEXT');
+
+# Check data state, after server-side data generation.
+check_data_state($node, 'client-side (text)');
+
+$node->pgbench(
+ '--initialize --init-steps=dtMg',
+ 0,
+ [qr{^$}],
+ [
+ qr{dropping old tables},
+ qr{creating tables},
+ qr{generating data \(client-side as multiple transactions},
+ qr{done in \d+\.\d\d s }
+ ],
+ 'pgbench --init-steps client-side TEXT');
+
+# Check data state, after server-side data generation.
+check_data_state($node, 'client-side (text)');
+
+
+# Test server-side generation with COPY BINARY
$node->pgbench(
- '--initialize --init-steps=dtC',
+ '--initialize --init-steps=dtc',
0,
[qr{^$}],
[
qr{dropping old tables},
qr{creating tables},
- qr{generating data \(client-side\)},
+ qr{generating data \(client-side as single transaction},
qr{done in \d+\.\d\d s }
],
'pgbench --init-steps client-side BINARY');
@@ -202,6 +237,52 @@ $node->pgbench(
# Check data state, after server-side data generation.
check_data_state($node, 'client-side (binary)');
+$node->pgbench(
+ '--initialize --init-steps=dtSc',
+ 0,
+ [qr{^$}],
+ [
+ qr{dropping old tables},
+ qr{creating tables},
+ qr{generating data \(client-side as single transaction},
+ qr{done in \d+\.\d\d s }
+ ],
+ 'pgbench --init-steps client-side BINARY');
+
+# Check data state, after server-side data generation.
+check_data_state($node, 'client-side (binary)');
+
+$node->pgbench(
+ '--initialize --init-steps=dtMc',
+ 0,
+ [qr{^$}],
+ [
+ qr{dropping old tables},
+ qr{creating tables},
+ qr{generating data \(client-side as multiple transactions},
+ qr{done in \d+\.\d\d s }
+ ],
+ 'pgbench --init-steps client-side BINARY');
+
+# Check data state, after server-side data generation.
+check_data_state($node, 'client-side (binary)');
+
+$node->pgbench(
+ '--initialize --init-steps=dtMc',
+ 0,
+ [qr{^$}],
+ [
+ qr{dropping old tables},
+ qr{creating tables},
+ qr{generating data \(client-side as multiple transactions},
+ qr{done in \d+\.\d\d s }
+ ],
+ 'pgbench --init-steps client-side BINARY');
+
+# Check data state, after server-side data generation.
+check_data_state($node, 'client-side (binary)');
+
+
# Run all builtin scripts, for a few transactions each
$node->pgbench(
'--transactions=5 -Dfoo=bla --client=2 --protocol=simple --builtin=t'
--
2.43.0
From 2471bb47bf715402ece61b084b69afe3bb53742e Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Fri, 6 Feb 2026 13:55:47 +0700
Subject: [PATCH v8 18/23] Removing commented out procedures
---
src/bin/pgbench/pgbench.c | 52 ---------------------------------------
1 file changed, 52 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index af71e358e71..34464b7037c 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -5206,35 +5206,6 @@ initGenerateDataClientSideText(PGconn *con)
}
-/*
- * Dumps binary buffer to file (purely for debugging)
- *
-static void
-dumpBufferToFile(char *filename)
-{
- FILE *file_ptr;
- size_t bytes_written;
-
- file_ptr = fopen(filename, "wb");
- if (file_ptr == NULL)
- {
- fprintf(stderr, "Error opening file %s\n", filename);
- return; // EXIT_FAILURE;
- }
-
- bytes_written = fwrite(bin_copy_buffer, 1, bin_copy_buffer_length, file_ptr);
-
- if (bytes_written != bin_copy_buffer_length)
- {
- fprintf(stderr, "Error writing to file or incomplete write\n");
- fclose(file_ptr);
- return; // EXIT_FAILURE;
- }
-
- fclose(file_ptr);
-}
- */
-
/*
* Save char data to buffer
*/
@@ -5277,17 +5248,6 @@ addColumnCounter(int16_t n)
bufferData((void *) &n, sizeof(n));
}
-/*
- * adds column with NULL value
- *
-static void
-addNullColumn()
-{
- int32_t null = -1;
- bufferData((void *) &null, sizeof(null));
-}
- */
-
/*
* adds column with inti32 value
*/
@@ -5312,18 +5272,6 @@ addInt64Column(int64_t value)
bufferData((void *) &data, sizeof(data));
}
-/*
- * adds column with char value
- *
-static void
-addCharColumn(char *value)
-{
- int32_t size = strlen(value);
- bufferData((void *) &size, sizeof(size));
- bufferCharData(value, size);
-}
- */
-
/*
* Starts communication with server for COPY FROM BINARY statement
*/
--
2.43.0
From 10dfa622b09c420d79da8774c9321f91540c8600 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Mon, 16 Feb 2026 09:08:17 +0700
Subject: [PATCH v8 19/23] Small changes due to big code review
---
src/bin/pgbench/pgbench.c | 71 +++++++++++++++++++++------------------
1 file changed, 39 insertions(+), 32 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 34464b7037c..9f3d7f99a25 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -873,7 +873,7 @@ static bool socket_has_input(socket_set *sa, int fd, int idx);
/* callback used to build rows for COPY during data loading */
typedef void (*initRowMethod) (PQExpBufferData *sql, int64 curr);
-typedef void (*initRowMethodBin) (PGconn *con, int64_t curr, int32_t parent);
+typedef void (*initRowMethodBinary) (PGconn *con, int64_t curr, int32_t parent, int8_t columnCounter);
/* callback functions for our flex lexer */
static const PsqlScanCallbacks pgbench_callbacks = {
@@ -942,7 +942,8 @@ usage(void)
" to generate data, server-side:\n"
" G: INSERT .. SELECT generate_series\n"
" U: INSERT .. SELECT unnest\n"
- " M: use multiple transactions to initialize data\n"
+ " S: flag to use single transaction to initialize data\n"
+ " M: flag to use multiple transactions to initialize data\n"
" v: invoke VACUUM on the standard tables\n"
" p: create primary key indexes on the standard tables\n"
" f: create foreign keys between the standard tables\n"
@@ -5053,8 +5054,8 @@ initAccount(PQExpBufferData *sql, int64 curr)
}
static void
-initPopulateTableText(PGconn *con, const char *table, int64 base,
- initRowMethod init_row)
+initPopulateTableCopyText(PGconn *con, const char *table, int64 base,
+ initRowMethod init_row)
{
int n;
int64 k;
@@ -5178,7 +5179,7 @@ initPopulateTableText(PGconn *con, const char *table, int64 base,
* a blank-padded string in pgbench_accounts.
*/
static void
-initGenerateDataClientSideText(PGconn *con)
+initGenerateDataClientSideTextFrmt(PGconn *con)
{
fprintf(stderr, "TEXT mode...\n");
@@ -5198,9 +5199,9 @@ initGenerateDataClientSideText(PGconn *con)
* fill branches, tellers, accounts in that order in case foreign keys
* already exist
*/
- initPopulateTableText(con, "pgbench_branches", nbranches, initBranch);
- initPopulateTableText(con, "pgbench_tellers", ntellers, initTeller);
- initPopulateTableText(con, "pgbench_accounts", naccounts, initAccount);
+ initPopulateTableCopyText(con, "pgbench_branches", nbranches, initBranch);
+ initPopulateTableCopyText(con, "pgbench_tellers", ntellers, initTeller);
+ initPopulateTableCopyText(con, "pgbench_accounts", naccounts, initAccount);
executeStatement(con, "commit");
}
@@ -5212,6 +5213,8 @@ initGenerateDataClientSideText(PGconn *con)
static void
bufferCharData(char *src, int32_t len)
{
+ Assert(bin_copy_buffer_length + len <= BIN_COPY_BUF_SIZE);
+
memcpy((char *) bin_copy_buffer + bin_copy_buffer_length, (char *) src, len);
bin_copy_buffer_length += len;
}
@@ -5227,13 +5230,16 @@ bufferData(void *src, int32_t len)
bufferCharData(src, len);
#else
if (len == 1)
+ {
bufferCharData(src, len);
- else
- for (int32_t i = 0; i < len; i++)
- {
- ((char *) bin_copy_buffer + bin_copy_buffer_length)[i] =
- ((char *) src)[len - i - 1];
- }
+ return;
+ }
+
+ for (int32_t i = 0; i < len; i++)
+ {
+ ((char *) bin_copy_buffer + bin_copy_buffer_length)[i] =
+ ((char *) src)[len - i - 1];
+ }
bin_copy_buffer_length += len;
#endif
@@ -5322,7 +5328,7 @@ flushBuffer(PGconn *con, int16_t row_len)
* Sends current branch row to buffer
*/
static void
-initBranchBinary(PGconn *con, int64_t curr, int32_t parent)
+initBranchBinary(PGconn *con, int64_t curr, int32_t parent, int8_t columnCounter)
{
/*
* Each row of branch table has following extra bytes:
@@ -5333,7 +5339,7 @@ initBranchBinary(PGconn *con, int64_t curr, int32_t parent)
flushBuffer(con, max_row_len);
- addColumnCounter(2);
+ addColumnCounter(columnCounter);
addInt32Column(curr + 1);
addInt32Column(0);
@@ -5343,7 +5349,7 @@ initBranchBinary(PGconn *con, int64_t curr, int32_t parent)
* Sends current teller row to buffer
*/
static void
-initTellerBinary(PGconn *con, int64_t curr, int32_t parent)
+initTellerBinary(PGconn *con, int64_t curr, int32_t parent, int8_t columnCounter)
{
/*
* Each row of tellers table has following extra bytes:
@@ -5354,7 +5360,7 @@ initTellerBinary(PGconn *con, int64_t curr, int32_t parent)
flushBuffer(con, max_row_len);
- addColumnCounter(3);
+ addColumnCounter(columnCounter);
addInt32Column(curr + 1);
addInt32Column(curr / parent + 1);
@@ -5365,7 +5371,7 @@ initTellerBinary(PGconn *con, int64_t curr, int32_t parent)
* Sends current account row to buffer
*/
static void
-initAccountBinary(PGconn *con, int64_t curr, int32_t parent)
+initAccountBinary(PGconn *con, int64_t curr, int32_t parent, int8_t columnCounter)
{
/*
* Each row of accounts table has following extra bytes:
@@ -5376,7 +5382,7 @@ initAccountBinary(PGconn *con, int64_t curr, int32_t parent)
flushBuffer(con, max_row_len);
- addColumnCounter(3);
+ addColumnCounter(columnCounter);
if (scale <= SCALE_32BIT_THRESHOLD)
addInt32Column(curr + 1);
@@ -5391,8 +5397,9 @@ initAccountBinary(PGconn *con, int64_t curr, int32_t parent)
* Universal wrapper for sending data in binary format
*/
static void
-initPopulateTableBinary(PGconn *con, char *table, char *columns,
- int counter, int64_t base, initRowMethodBin init_row)
+initPopulateTableCopyBinary(PGconn *con, char *table, char *columns,
+ int counter, int64_t base, initRowMethodBinary init_row,
+ int columnCounter)
{
int n;
PGresult *res;
@@ -5426,7 +5433,7 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
for (int64_t i = start; i < start + base; i++)
{
- init_row(con, i, base);
+ init_row(con, i, base, columnCounter);
}
res = PQgetResult(con);
@@ -5460,7 +5467,7 @@ initPopulateTableBinary(PGconn *con, char *table, char *columns,
* Wrapper for binary data load
*/
static void
-initGenerateDataClientSideBinary(PGconn *con)
+initGenerateDataClientSideBinaryFrmt(PGconn *con)
{
fprintf(stderr, "BINARY mode...\n");
@@ -5486,12 +5493,12 @@ initGenerateDataClientSideBinary(PGconn *con)
if (multi_xact)
executeStatement(con, "begin");
- initPopulateTableBinary(con, "pgbench_branches", "bid, bbalance",
- i, nbranches, initBranchBinary);
- initPopulateTableBinary(con, "pgbench_tellers", "tid, bid, tbalance",
- i, ntellers, initTellerBinary);
- initPopulateTableBinary(con, "pgbench_accounts", "aid, bid, abalance",
- i, naccounts, initAccountBinary);
+ initPopulateTableCopyBinary(con, "pgbench_branches", "bid, bbalance",
+ i, nbranches, initBranchBinary, 2);
+ initPopulateTableCopyBinary(con, "pgbench_tellers", "tid, bid, tbalance",
+ i, ntellers, initTellerBinary, 3);
+ initPopulateTableCopyBinary(con, "pgbench_accounts", "aid, bid, abalance",
+ i, naccounts, initAccountBinary, 3);
if (multi_xact)
executeStatement(con, "commit");
@@ -5515,10 +5522,10 @@ initGenerateDataClientSide(PGconn *con)
switch (data_generation_type)
{
case INIT_STEP_GEN_TYPE_COPY_TEXT:
- initGenerateDataClientSideText(con);
+ initGenerateDataClientSideTextFrmt(con);
break;
case INIT_STEP_GEN_TYPE_COPY_BINARY:
- initGenerateDataClientSideBinary(con);
+ initGenerateDataClientSideBinaryFrmt(con);
break;
}
}
--
2.43.0
From d95c53d412ff0e94e162eaf3990233fae007ae3f Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Mon, 16 Feb 2026 10:07:13 +0700
Subject: [PATCH v8 20/23] Running indent to fix code formatting
---
src/bin/pgbench/pgbench.c | 111 +++++++++++++++++++++-----------------
1 file changed, 62 insertions(+), 49 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 9f3d7f99a25..8198f1d4c9a 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -159,7 +159,7 @@ typedef struct socket_set
/********************************************************************
* some configurable parameters */
-#define DEFAULT_INIT_STEPS "dtgvp" /* default -I setting */
+#define DEFAULT_INIT_STEPS "dtgvp" /* default -I setting */
#define ALL_INIT_STEPS "dtgMScGUvpf" /* all possible steps */
#define LOG_STEP_SECONDS 5 /* seconds between log messages */
@@ -171,16 +171,24 @@ typedef struct socket_set
#define MAX_ZIPFIAN_PARAM 1000.0 /* maximum parameter for zipfian */
/* server-side methods to generate data */
-#define INIT_STEP_GEN_TYPE_INSERT_SERIES 'G' /* use INSERT .. SELECT generate_series to generate data */
-#define INIT_STEP_GEN_TYPE_INSERT_UNNEST 'U' /* use INSERT .. SELECT unnest to generate data */
+#define INIT_STEP_GEN_TYPE_INSERT_SERIES 'G' /* use INSERT .. SELECT
+ * generate_series to generate
+ * data */
+#define INIT_STEP_GEN_TYPE_INSERT_UNNEST 'U' /* use INSERT .. SELECT unnest
+ * to generate data */
/* client-side methods to generate data */
-#define INIT_STEP_GEN_TYPE_COPY_TEXT 'g' /* use COPY .. FROM STDIN .. TEXT to generate data */
-#define INIT_STEP_GEN_TYPE_COPY_BINARY 'c' /* use COPY .. FROM STDIN .. BINARY to generate data */
+#define INIT_STEP_GEN_TYPE_COPY_TEXT 'g' /* use COPY .. FROM STDIN ..
+ * TEXT to generate data */
+#define INIT_STEP_GEN_TYPE_COPY_BINARY 'c' /* use COPY .. FROM STDIN ..
+ * BINARY to generate data */
/* data init pseudo steps */
-#define INIT_STEP_GEN_TYPE_SINGLE_XACT 'S' /* switch to init data as single transaction */
-#define INIT_STEP_GEN_TYPE_MULTI_XACT 'M' /* switch to init data as multiple transactions */
+#define INIT_STEP_GEN_TYPE_SINGLE_XACT 'S' /* switch to init data as
+ * single transaction */
+#define INIT_STEP_GEN_TYPE_MULTI_XACT 'M' /* switch to init data as
+ * multiple transactions */
-static bool multi_xact = false; /* init data type (as single or multiple transactions) */
+static bool multi_xact = false; /* init data type (as single or multiple
+ * transactions) */
static int nxacts = 0; /* number of transactions per client */
static int duration = 0; /* duration in seconds */
static int64 end_time = 0; /* when to stop in micro seconds, under -T */
@@ -194,14 +202,15 @@ static int scale = 1;
/*
* mode of data generation to use
*/
-static char data_generation_type = INIT_STEP_GEN_TYPE_COPY_TEXT;
+static char data_generation_type = INIT_STEP_GEN_TYPE_COPY_TEXT;
/*
* COPY FROM BINARY execution buffer
*/
-#define BIN_COPY_BUF_SIZE 102400 /* maximum buffer size for COPY FROM BINARY */
-static char *bin_copy_buffer = NULL; /* buffer for COPY FROM BINARY */
-static int32_t bin_copy_buffer_length = 0; /* current buffer size */
+#define BIN_COPY_BUF_SIZE 102400 /* maximum buffer size for COPY FROM
+ * BINARY */
+static char *bin_copy_buffer = NULL; /* buffer for COPY FROM BINARY */
+static int32_t bin_copy_buffer_length = 0; /* current buffer size */
/*
* fillfactor. for example, fillfactor = 90 will use only 90 percent
@@ -872,8 +881,8 @@ static int wait_on_socket_set(socket_set *sa, int64 usecs);
static bool socket_has_input(socket_set *sa, int fd, int idx);
/* callback used to build rows for COPY during data loading */
-typedef void (*initRowMethod) (PQExpBufferData *sql, int64 curr);
-typedef void (*initRowMethodBinary) (PGconn *con, int64_t curr, int32_t parent, int8_t columnCounter);
+typedef void (*initRowMethod) (PQExpBufferData *sql, int64 curr);
+typedef void (*initRowMethodBinary) (PGconn *con, int64_t curr, int32_t parent, int8_t columnCounter);
/* callback functions for our flex lexer */
static const PsqlScanCallbacks pgbench_callbacks = {
@@ -5200,7 +5209,7 @@ initGenerateDataClientSideTextFrmt(PGconn *con)
* already exist
*/
initPopulateTableCopyText(con, "pgbench_branches", nbranches, initBranch);
- initPopulateTableCopyText(con, "pgbench_tellers", ntellers, initTeller);
+ initPopulateTableCopyText(con, "pgbench_tellers", ntellers, initTeller);
initPopulateTableCopyText(con, "pgbench_accounts", naccounts, initAccount);
executeStatement(con, "commit");
@@ -5260,8 +5269,9 @@ addColumnCounter(int16_t n)
static void
addInt32Column(int32_t value)
{
- int32_t data = value;
- int32_t size = sizeof(data);
+ int32_t data = value;
+ int32_t size = sizeof(data);
+
bufferData((void *) &size, sizeof(size));
bufferData((void *) &data, sizeof(data));
}
@@ -5272,8 +5282,9 @@ addInt32Column(int32_t value)
static void
addInt64Column(int64_t value)
{
- int64_t data = value;
- int32_t size = sizeof(data);
+ int64_t data = value;
+ int32_t size = sizeof(data);
+
bufferData((void *) &size, sizeof(size));
bufferData((void *) &data, sizeof(data));
}
@@ -5284,9 +5295,9 @@ addInt64Column(int64_t value)
static void
sendBinaryCopyHeader(PGconn *con)
{
- char header[] = {'P','G','C','O','P','Y','\n','\377','\r','\n','\0',
- '\0','\0','\0','\0',
- '\0','\0','\0','\0' };
+ char header[] = {'P', 'G', 'C', 'O', 'P', 'Y', '\n', '\377', '\r', '\n', '\0',
+ '\0', '\0', '\0', '\0',
+ '\0', '\0', '\0', '\0'};
PQputCopyData(con, header, 19);
}
@@ -5297,7 +5308,7 @@ sendBinaryCopyHeader(PGconn *con)
static void
sendBinaryCopyTrailer(PGconn *con)
{
- static char trailer[] = { 0xFF, 0xFF };
+ static char trailer[] = {0xFF, 0xFF};
PQputCopyData(con, trailer, 2);
}
@@ -5308,7 +5319,7 @@ sendBinaryCopyTrailer(PGconn *con)
static void
flushBuffer(PGconn *con, int16_t row_len)
{
- PGresult *res;
+ PGresult *res;
if (bin_copy_buffer_length + row_len > BIN_COPY_BUF_SIZE)
{
@@ -5330,12 +5341,13 @@ flushBuffer(PGconn *con, int16_t row_len)
static void
initBranchBinary(PGconn *con, int64_t curr, int32_t parent, int8_t columnCounter)
{
- /*
+ /*---
* Each row of branch table has following extra bytes:
* - 2 bytes for number of columns
* - 4 bytes as length for each of table's 3 columns
+ *---
*/
- int16_t max_row_len = 35 + 2 + 4*3; /* max row size is 35 */
+ int16_t max_row_len = 35 + 2 + 4 * 3; /* max row size is 35 */
flushBuffer(con, max_row_len);
@@ -5351,12 +5363,13 @@ initBranchBinary(PGconn *con, int64_t curr, int32_t parent, int8_t columnCounter
static void
initTellerBinary(PGconn *con, int64_t curr, int32_t parent, int8_t columnCounter)
{
- /*
+ /*---
* Each row of tellers table has following extra bytes:
* - 2 bytes for number of columns
* - 4 bytes as length for each of table's 4 columns
+ *---
*/
- int16_t max_row_len = 40 + 2 + 4*4; /* max row size is 40 */
+ int16_t max_row_len = 40 + 2 + 4 * 4; /* max row size is 40 */
flushBuffer(con, max_row_len);
@@ -5373,12 +5386,13 @@ initTellerBinary(PGconn *con, int64_t curr, int32_t parent, int8_t columnCounter
static void
initAccountBinary(PGconn *con, int64_t curr, int32_t parent, int8_t columnCounter)
{
- /*
+ /*---
* Each row of accounts table has following extra bytes:
* - 2 bytes for number of columns
* - 4 bytes as length for each of table's 4 columns
*/
- int16_t max_row_len = 250 + 2 + 4*4; /* max row size is 250 for int64 */
+ int16_t max_row_len = 250 + 2 + 4 * 4; /* max row size is 250 for
+ * int64 */
flushBuffer(con, max_row_len);
@@ -5401,11 +5415,11 @@ initPopulateTableCopyBinary(PGconn *con, char *table, char *columns,
int counter, int64_t base, initRowMethodBinary init_row,
int columnCounter)
{
- int n;
- PGresult *res;
- char copy_statement[256];
- const char *copy_statement_fmt = "copy %s (%s) from stdin (format binary)";
- int64_t start = base * counter;
+ int n;
+ PGresult *res;
+ char copy_statement[256];
+ const char *copy_statement_fmt = "copy %s (%s) from stdin (format binary)";
+ int64_t start = base * counter;
bin_copy_buffer_length = 0;
@@ -5450,7 +5464,7 @@ initPopulateTableCopyBinary(PGconn *con, char *table, char *columns,
res = PQgetResult(con);
if (PQresultStatus(res) == PGRES_COPY_IN)
{
- if (PQputCopyEnd(con, NULL) == 1) /* success */
+ if (PQputCopyEnd(con, NULL) == 1) /* success */
{
PQclear(res);
res = PQgetResult(con);
@@ -5476,9 +5490,8 @@ initGenerateDataClientSideBinaryFrmt(PGconn *con)
bin_copy_buffer_length = 0;
/*
- * we do all of this in multiple transactions
- * to minimize load on DB server and perhaps
- * in future allow load in parallel sessions
+ * we do all of this in multiple transactions to minimize load on DB
+ * server and perhaps in future allow load in parallel sessions
*/
executeStatement(con, "begin");
@@ -5495,8 +5508,8 @@ initGenerateDataClientSideBinaryFrmt(PGconn *con)
initPopulateTableCopyBinary(con, "pgbench_branches", "bid, bbalance",
i, nbranches, initBranchBinary, 2);
- initPopulateTableCopyBinary(con, "pgbench_tellers", "tid, bid, tbalance",
- i, ntellers, initTellerBinary, 3);
+ initPopulateTableCopyBinary(con, "pgbench_tellers", "tid, bid, tbalance",
+ i, ntellers, initTellerBinary, 3);
initPopulateTableCopyBinary(con, "pgbench_accounts", "aid, bid, abalance",
i, naccounts, initAccountBinary, 3);
@@ -5570,10 +5583,10 @@ generateDataInsertSeries(PGconn *con)
printfPQExpBuffer(&sql,
"insert into pgbench_accounts(aid, bid, abalance, "
- "filler) "
+ "filler) "
"select aid + 1, aid / %d + 1, 0, '' "
"from generate_series(" INT64_FORMAT ", "
- INT64_FORMAT ") as aid",
+ INT64_FORMAT ") as aid",
naccounts, (int64) i * naccounts,
(int64) (i + 1) * naccounts - 1);
executeStatement(con, sql.data);
@@ -5622,7 +5635,7 @@ generateDataInsertUnnest(PGconn *con)
printfPQExpBuffer(&sql,
"insert into pgbench_tellers(tid, bid, tbalance) "
"select unnest(array_agg(s.i order by s.i)) as tid, "
- "%d as bid, 0 as tbalance "
+ "%d as bid, 0 as tbalance "
"from generate_series(%d, %d) as s(i)",
s + 1, s * ntellers + 1, (s + 1) * ntellers);
executeStatement(con, sql.data);
@@ -5630,13 +5643,13 @@ generateDataInsertUnnest(PGconn *con)
printfPQExpBuffer(&sql,
"with data as ("
" select generate_series(" INT64_FORMAT ", "
- INT64_FORMAT ") as i) "
+ INT64_FORMAT ") as i) "
"insert into pgbench_accounts(aid, bid, "
- "abalance, filler) "
+ "abalance, filler) "
"select unnest(aid), unnest(bid), 0 as abalance, "
- "'' as filler "
+ "'' as filler "
"from (select array_agg(i+1) aid, "
- "array_agg(i/%d + 1) bid from data)",
+ "array_agg(i/%d + 1) bid from data)",
(int64) s * naccounts + 1,
(int64) (s + 1) * naccounts, naccounts);
executeStatement(con, sql.data);
@@ -5758,7 +5771,7 @@ initCreateFKeys(PGconn *con)
static void
checkInitSteps(const char *initialize_steps)
{
- char data_init_type = 0;
+ char data_init_type = 0;
if (initialize_steps[0] == '\0')
pg_fatal("no initialization steps specified");
--
2.43.0
From 270d2bf14b6873ff66fc37ca0a5dccc055dab1f7 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Mon, 16 Feb 2026 14:57:18 +0700
Subject: [PATCH v8 21/23] Removing debugging filler value
---
src/bin/pgbench/pgbench.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 8198f1d4c9a..710b2a5b70d 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -4954,8 +4954,8 @@ initCreateTables(PGconn *con)
static const struct ddlinfo DDLs[] = {
{
"pgbench_history",
- "tid int,bid int,aid int,delta int,mtime timestamp,filler char(22) default '?'",
- "tid int,bid int,aid bigint,delta int,mtime timestamp,filler char(22) default '?'",
+ "tid int,bid int,aid int,delta int,mtime timestamp,filler char(22) default ''",
+ "tid int,bid int,aid bigint,delta int,mtime timestamp,filler char(22) default ''",
0
},
{
@@ -4966,8 +4966,8 @@ initCreateTables(PGconn *con)
},
{
"pgbench_accounts",
- "aid int not null,bid int,abalance int,filler char(84) default '?'",
- "aid bigint not null,bid int,abalance int,filler char(84) default '?'",
+ "aid int not null,bid int,abalance int,filler char(84) default ''",
+ "aid bigint not null,bid int,abalance int,filler char(84) default ''",
1
},
{
--
2.43.0
From 86a111c63e7006d4801ce7bb594a5b77d87b42b0 Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Mon, 23 Feb 2026 16:15:06 +0700
Subject: [PATCH v8 22/23] Moving out logic showing data load progress into own
procedure
---
src/bin/pgbench/pgbench.c | 142 ++++++++++++++++++++------------------
1 file changed, 75 insertions(+), 67 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index 710b2a5b70d..e3e0e073792 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -487,6 +487,9 @@ typedef struct StatsData
*/
static pg_time_usec_t epoch_shift;
+/* used to track elapsed time and estimate of the remaining time of data load */
+static pg_time_usec_t data_load_start;
+
/*
* Error status for errors during script execution.
*/
@@ -5062,27 +5065,82 @@ initAccount(PQExpBufferData *sql, int64 curr)
curr + 1, curr / naccounts + 1);
}
+static void
+showPopulateTableCopyProgress(const char *table, int64 current, int64 total)
+{
+ static int chars = 0;
+ static int prev_chars = 0;
+ static int log_interval = 1;
+
+ /* Stay on the same line if reporting to a terminal */
+ char eol = isatty(fileno(stderr)) ? '\r' : '\n';
+
+ double elapsed_sec = PG_TIME_GET_DOUBLE(pg_time_now() - data_load_start);
+ double remaining_sec = ((double) total - current) * elapsed_sec / current;
+
+ /*
+ * If we want to stick with the original logging, print a message each
+ * 100k inserted rows.
+ */
+ if ((!use_quiet) && (current % 100000 == 0))
+ {
+ chars = fprintf(stderr, INT64_FORMAT " of " INT64_FORMAT " tuples (%d%%) of %s done (elapsed %.2f s, remaining %.2f s)",
+ current, total,
+ (int) ((current * 100) / total),
+ table, elapsed_sec, remaining_sec);
+
+ /*
+ * If the previous progress message is longer than the current one,
+ * add spaces to the current line to fully overwrite any remaining
+ * characters from the previous message.
+ */
+ if (prev_chars > chars)
+ fprintf(stderr, "%*c", prev_chars - chars, ' ');
+ fputc(eol, stderr);
+ prev_chars = chars;
+ }
+ /* let's not call the timing for each row, but only each 100 rows */
+ else if (use_quiet && (current % 100 == 0))
+ {
+ /* have we reached the next interval (or end)? */
+ if ((current == total) || (elapsed_sec >= log_interval * LOG_STEP_SECONDS))
+ {
+ chars = fprintf(stderr, INT64_FORMAT " of " INT64_FORMAT " tuples (%d%%) of %s done (elapsed %.2f s, remaining %.2f s)",
+ current, total,
+ (int) ((current * 100) / total),
+ table, elapsed_sec, remaining_sec);
+
+ /*
+ * If the previous progress message is longer than the current
+ * one, add spaces to the current line to fully overwrite any
+ * remaining characters from the previous message.
+ */
+ if (prev_chars > chars)
+ fprintf(stderr, "%*c", prev_chars - chars, ' ');
+ fputc(eol, stderr);
+ prev_chars = chars;
+
+ /* skip to the next interval */
+ log_interval = (int) ceil(elapsed_sec / LOG_STEP_SECONDS);
+ }
+ }
+
+ if (current == total && chars != 0 && eol != '\n')
+ fprintf(stderr, "%*c\r", chars, ' '); /* Clear the current line */
+}
+
static void
initPopulateTableCopyText(PGconn *con, const char *table, int64 base,
initRowMethod init_row)
{
int n;
int64 k;
- int chars = 0;
- int prev_chars = 0;
PGresult *res;
PQExpBufferData sql;
char copy_statement[256];
const char *copy_statement_fmt = "copy %s from stdin";
int64 total = base * scale;
- /* used to track elapsed time and estimate of the remaining time */
- pg_time_usec_t start;
- int log_interval = 1;
-
- /* Stay on the same line if reporting to a terminal */
- char eol = isatty(fileno(stderr)) ? '\r' : '\n';
-
initPQExpBuffer(&sql);
/* Use COPY with FREEZE on v14 and later for all ordinary tables */
@@ -5103,12 +5161,8 @@ initPopulateTableCopyText(PGconn *con, const char *table, int64 base,
pg_fatal("unexpected copy in result: %s", PQerrorMessage(con));
PQclear(res);
- start = pg_time_now();
-
for (k = 0; k < total; k++)
{
- int64 j = k + 1;
-
init_row(&sql, k);
if (PQputline(con, sql.data))
pg_fatal("PQputline failed");
@@ -5116,62 +5170,9 @@ initPopulateTableCopyText(PGconn *con, const char *table, int64 base,
if (CancelRequested)
break;
- /*
- * If we want to stick with the original logging, print a message each
- * 100k inserted rows.
- */
- if ((!use_quiet) && (j % 100000 == 0))
- {
- double elapsed_sec = PG_TIME_GET_DOUBLE(pg_time_now() - start);
- double remaining_sec = ((double) total - j) * elapsed_sec / j;
-
- chars = fprintf(stderr, INT64_FORMAT " of " INT64_FORMAT " tuples (%d%%) of %s done (elapsed %.2f s, remaining %.2f s)",
- j, total,
- (int) ((j * 100) / total),
- table, elapsed_sec, remaining_sec);
-
- /*
- * If the previous progress message is longer than the current
- * one, add spaces to the current line to fully overwrite any
- * remaining characters from the previous message.
- */
- if (prev_chars > chars)
- fprintf(stderr, "%*c", prev_chars - chars, ' ');
- fputc(eol, stderr);
- prev_chars = chars;
- }
- /* let's not call the timing for each row, but only each 100 rows */
- else if (use_quiet && (j % 100 == 0))
- {
- double elapsed_sec = PG_TIME_GET_DOUBLE(pg_time_now() - start);
- double remaining_sec = ((double) total - j) * elapsed_sec / j;
-
- /* have we reached the next interval (or end)? */
- if ((j == total) || (elapsed_sec >= log_interval * LOG_STEP_SECONDS))
- {
- chars = fprintf(stderr, INT64_FORMAT " of " INT64_FORMAT " tuples (%d%%) of %s done (elapsed %.2f s, remaining %.2f s)",
- j, total,
- (int) ((j * 100) / total),
- table, elapsed_sec, remaining_sec);
-
- /*
- * If the previous progress message is longer than the current
- * one, add spaces to the current line to fully overwrite any
- * remaining characters from the previous message.
- */
- if (prev_chars > chars)
- fprintf(stderr, "%*c", prev_chars - chars, ' ');
- fputc(eol, stderr);
- prev_chars = chars;
-
- /* skip to the next interval */
- log_interval = (int) ceil(elapsed_sec / LOG_STEP_SECONDS);
- }
- }
+ showPopulateTableCopyProgress(table, k + 1, total);
}
- if (chars != 0 && eol != '\n')
- fprintf(stderr, "%*c\r", chars, ' '); /* Clear the current line */
if (PQputline(con, "\\.\n"))
pg_fatal("very last PQputline failed");
@@ -5448,6 +5449,11 @@ initPopulateTableCopyBinary(PGconn *con, char *table, char *columns,
for (int64_t i = start; i < start + base; i++)
{
init_row(con, i, base, columnCounter);
+
+ if (CancelRequested)
+ break;
+
+ showPopulateTableCopyProgress(table, i, base * scale);
}
res = PQgetResult(con);
@@ -5532,6 +5538,8 @@ initGenerateDataClientSide(PGconn *con)
fprintf(stderr, "generating data (client-side as %s transaction%s) in ",
multi_xact ? "multiple" : "single", multi_xact ? "s" : "");
+ data_load_start = pg_time_now();
+
switch (data_generation_type)
{
case INIT_STEP_GEN_TYPE_COPY_TEXT:
--
2.43.0
From afe3f69b23190f375b57d5cf0a3eae2cd8e40caa Mon Sep 17 00:00:00 2001
From: Boris Mironov <[email protected]>
Date: Mon, 23 Feb 2026 17:07:16 +0700
Subject: [PATCH v8 23/23] Adding multi-transaction mode to client-side data
generation
---
src/bin/pgbench/pgbench.c | 52 ++++++++++++--------
src/bin/pgbench/t/001_pgbench_with_server.pl | 47 +++++++++---------
2 files changed, 56 insertions(+), 43 deletions(-)
diff --git a/src/bin/pgbench/pgbench.c b/src/bin/pgbench/pgbench.c
index e3e0e073792..1b73da904ad 100644
--- a/src/bin/pgbench/pgbench.c
+++ b/src/bin/pgbench/pgbench.c
@@ -5130,22 +5130,22 @@ showPopulateTableCopyProgress(const char *table, int64 current, int64 total)
}
static void
-initPopulateTableCopyText(PGconn *con, const char *table, int64 base,
+initPopulateTableCopyText(PGconn *con, const char *table, int counter, int64 base,
initRowMethod init_row)
{
int n;
- int64 k;
PGresult *res;
PQExpBufferData sql;
char copy_statement[256];
const char *copy_statement_fmt = "copy %s from stdin";
- int64 total = base * scale;
+ int64 start = base * counter;
initPQExpBuffer(&sql);
/* Use COPY with FREEZE on v14 and later for all ordinary tables */
if ((PQserverVersion(con) >= 140000) &&
- get_table_relkind(con, table) == RELKIND_RELATION)
+ get_table_relkind(con, table) == RELKIND_RELATION &&
+ !multi_xact)
copy_statement_fmt = "copy %s from stdin with (freeze on)";
@@ -5161,16 +5161,16 @@ initPopulateTableCopyText(PGconn *con, const char *table, int64 base,
pg_fatal("unexpected copy in result: %s", PQerrorMessage(con));
PQclear(res);
- for (k = 0; k < total; k++)
+ for (int64_t i = start; i < start + base; i++)
{
- init_row(&sql, k);
+ init_row(&sql, i);
if (PQputline(con, sql.data))
pg_fatal("PQputline failed");
if (CancelRequested)
break;
- showPopulateTableCopyProgress(table, k + 1, total);
+ showPopulateTableCopyProgress(table, i, base * scale);
}
@@ -5193,9 +5193,6 @@ initGenerateDataClientSideTextFrmt(PGconn *con)
{
fprintf(stderr, "TEXT mode...\n");
- if (multi_xact)
- fprintf(stderr, "WARNING! Multiple transactions are not supported in this mode\n");
-
/*
* we do all of this in one transaction to enable the backend's
* data-loading optimizations
@@ -5205,15 +5202,28 @@ initGenerateDataClientSideTextFrmt(PGconn *con)
/* truncate away any old data */
initTruncateTables(con);
- /*
- * fill branches, tellers, accounts in that order in case foreign keys
- * already exist
- */
- initPopulateTableCopyText(con, "pgbench_branches", nbranches, initBranch);
- initPopulateTableCopyText(con, "pgbench_tellers", ntellers, initTeller);
- initPopulateTableCopyText(con, "pgbench_accounts", naccounts, initAccount);
+ if (multi_xact)
+ executeStatement(con, "commit");
+
+ for (int i = 0; i < scale; i++)
+ {
+ if (multi_xact)
+ executeStatement(con, "begin");
+
+ /*
+ * fill branches, tellers, accounts in that order in case foreign keys
+ * already exist
+ */
+ initPopulateTableCopyText(con, "pgbench_branches", i, nbranches, initBranch);
+ initPopulateTableCopyText(con, "pgbench_tellers", i, ntellers, initTeller);
+ initPopulateTableCopyText(con, "pgbench_accounts", i, naccounts, initAccount);
- executeStatement(con, "commit");
+ if (multi_xact)
+ executeStatement(con, "commit");
+ }
+
+ if (!multi_xact)
+ executeStatement(con, "commit");
}
@@ -5426,9 +5436,9 @@ initPopulateTableCopyBinary(PGconn *con, char *table, char *columns,
/* Use COPY with FREEZE on v14 and later for all ordinary tables */
if ((PQserverVersion(con) >= 140000) &&
- get_table_relkind(con, table) == RELKIND_RELATION)
- if (!multi_xact)
- copy_statement_fmt = "copy %s (%s) from stdin with (format binary, freeze on)";
+ get_table_relkind(con, table) == RELKIND_RELATION &&
+ !multi_xact)
+ copy_statement_fmt = "copy %s (%s) from stdin with (format binary, freeze on)";
n = pg_snprintf(copy_statement, sizeof(copy_statement), copy_statement_fmt, table, columns);
if (n >= sizeof(copy_statement))
diff --git a/src/bin/pgbench/t/001_pgbench_with_server.pl b/src/bin/pgbench/t/001_pgbench_with_server.pl
index acb1e31d3e8..46569b0eb37 100644
--- a/src/bin/pgbench/t/001_pgbench_with_server.pl
+++ b/src/bin/pgbench/t/001_pgbench_with_server.pl
@@ -189,7 +189,7 @@ $node->pgbench(
check_data_state($node, 'server-side (unnest)');
-# Test server-side generation with COPY TEXT
+# Test client-side generation with COPY TEXT
$node->pgbench(
'--initialize --init-steps=dtg',
0,
@@ -200,45 +200,48 @@ $node->pgbench(
qr{generating data \(client-side as single transaction},
qr{done in \d+\.\d\d s }
],
- 'pgbench --init-steps client-side TEXT');
+ 'pgbench --init-steps client-side TEXT (single XACT #1)');
-# Check data state, after server-side data generation.
+# Check data state, after client-side data generation.
check_data_state($node, 'client-side (text)');
$node->pgbench(
- '--initialize --init-steps=dtMg',
+ '--initialize --init-steps=dtSg',
0,
[qr{^$}],
[
qr{dropping old tables},
qr{creating tables},
- qr{generating data \(client-side as multiple transactions},
+ qr{generating data \(client-side as single transaction},
+ qr{\d of \d+ tuples \(\d%\) of pgbench_branches done},
+ qr{\d of \d+ tuples \(\d%\) of pgbench_tellers done},
+ qr{\d of \d+ tuples \(\d%\) of pgbench_accounts done},
qr{done in \d+\.\d\d s }
],
- 'pgbench --init-steps client-side TEXT');
+ 'pgbench --init-steps client-side TEXT (single XACT #2)');
-# Check data state, after server-side data generation.
+# Check data state, after client-side data generation.
check_data_state($node, 'client-side (text)');
-
-# Test server-side generation with COPY BINARY
$node->pgbench(
- '--initialize --init-steps=dtc',
+ '--initialize --init-steps=dtMg',
0,
[qr{^$}],
[
qr{dropping old tables},
qr{creating tables},
- qr{generating data \(client-side as single transaction},
+ qr{generating data \(client-side as multiple transactions},
qr{done in \d+\.\d\d s }
],
- 'pgbench --init-steps client-side BINARY');
+ 'pgbench --init-steps client-side TEXT (multiple XACTs)');
+
+# Check data state, after client-side data generation.
+check_data_state($node, 'client-side (text)');
-# Check data state, after server-side data generation.
-check_data_state($node, 'client-side (binary)');
+# Test client-side generation with COPY BINARY
$node->pgbench(
- '--initialize --init-steps=dtSc',
+ '--initialize --init-steps=dtc',
0,
[qr{^$}],
[
@@ -247,24 +250,24 @@ $node->pgbench(
qr{generating data \(client-side as single transaction},
qr{done in \d+\.\d\d s }
],
- 'pgbench --init-steps client-side BINARY');
+ 'pgbench --init-steps client-side BINARY (single XACT #1)');
-# Check data state, after server-side data generation.
+# Check data state, after client-side data generation.
check_data_state($node, 'client-side (binary)');
$node->pgbench(
- '--initialize --init-steps=dtMc',
+ '--initialize --init-steps=dtSc',
0,
[qr{^$}],
[
qr{dropping old tables},
qr{creating tables},
- qr{generating data \(client-side as multiple transactions},
+ qr{generating data \(client-side as single transaction},
qr{done in \d+\.\d\d s }
],
- 'pgbench --init-steps client-side BINARY');
+ 'pgbench --init-steps client-side BINARY (single XACT #2)');
-# Check data state, after server-side data generation.
+# Check data state, after client-side data generation.
check_data_state($node, 'client-side (binary)');
$node->pgbench(
@@ -279,7 +282,7 @@ $node->pgbench(
],
'pgbench --init-steps client-side BINARY');
-# Check data state, after server-side data generation.
+# Check data state, after client-side data generation.
check_data_state($node, 'client-side (binary)');
--
2.43.0
^ permalink raw reply [nested|flat] 98+ messages in thread
end of thread, other threads:[~2026-02-23 10:41 UTC | newest]
Thread overview: 98+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2019-05-28 18:16 [PATCH v29 4/6] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v28 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v27 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v25 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v27 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v27 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v16 4/7] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v24 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v25 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v24 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v27 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v28 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v25 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v27 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v25 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v27 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v27 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v25 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v28 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v28 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v25 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v24 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v25 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v25 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v25 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v27 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v25 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v24 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v24 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v24 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v27 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v25 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v28 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v24 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v27 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v28 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v24 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v17 4/7] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v15 4/7] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v16 4/7] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v24 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v27 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v25 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v28 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v25 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v24 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v27 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH 4/6] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v30 4/6] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v27 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v28 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v25 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v27 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v28 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v28 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v27 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v28 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v28 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v24 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v24 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v25 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH 4/6] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v19 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v20 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH 4/6] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v24 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v28 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v28 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v27 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v25 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v28 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v25 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v27 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v25 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v28 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v24 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v28 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v28 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v24 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v14 4/7] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v23 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v25 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v24 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v27 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v24 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v27 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v24 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v24 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v25 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v24 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH v28 3/5] Track collation versions for indexes. Thomas Munro <[email protected]>
2019-05-28 18:16 [PATCH 4/6] Track collation versions for indexes. Thomas Munro <[email protected]>
2026-01-30 13:22 RE: Idea to enhance pgbench by more modes to generate data (multi-TXNs, UNNEST, COPY BINARY) Madyshev Egor <[email protected]>
2026-01-30 14:47 ` Re: Idea to enhance pgbench by more modes to generate data (multi-TXNs, UNNEST, COPY BINARY) Boris Mironov <[email protected]>
2026-02-06 10:48 ` Re: Idea to enhance pgbench by more modes to generate data (multi-TXNs, UNNEST, COPY BINARY) Boris Mironov <[email protected]>
2026-02-06 11:08 ` Re: Idea to enhance pgbench by more modes to generate data (multi-TXNs, UNNEST, COPY BINARY) Boris Mironov <[email protected]>
2026-02-09 13:19 ` RE: Idea to enhance pgbench by more modes to generate data (multi-TXNs, UNNEST, COPY BINARY) Madyshev Egor <[email protected]>
2026-02-23 10:41 ` Re: Idea to enhance pgbench by more modes to generate data (multi-TXNs, UNNEST, COPY BINARY) Boris Mironov <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox